AI RESEARCH
A Reality Check of Language Models as Formalizers on Constraint Satisfaction Problems
arXiv CS.CL
•
ArXi:2505.13252v4 Announce Type: replace Recent work shows superior performance when using large language models (LLMs) as formalizers instead of as end-to-end solvers for symbolic reasoning problems. Given the problem description, the LLM generates a formal program that derives a solution via an external solver. We systematically investigate the formalization capability of LLMs on real-life constraint satisfaction problems on 4 benchmarks, 6 LLMs, and 2 types of formal languages.