Reproduce “Formally Solving Answer-Construction Problems in Lean” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jialiang Sun, Yuzhi Tang, Ao Li, Chris J. Maddison, Kuldeep S. Meel
The Enumerate-Conjecture-Prove (ECP) neuro-symbolic framework formally solves answer-construction problems in Lean by integrating general LLMs for answer enumeration and conjecture with specialized prover LLMs for formal proof generation, alongside an admissibility checking mechanism. This approach led to an increase in problem-solving accuracy from 17.3% to 24.0% on MathArena and a reduction in computational cost by over 60% compared to LLM-only baselines.
Key insights
- The explicit division of labor, leveraging general LLMs for exploration and specialized prover LLMs for rigorous verification, significantly improves both accuracy and cost-efficiency for formal mathematical problem-solving.
- Admissibility checking is a critical component for practical formal reasoning, preventing LLMs from generating logically valid but practically trivial or circular answers that bypass the actual problem-solving task.
- Bounded enumeration, facilitated by general LLMs with tool-calling capabilities, proves essential for constructing correct answers, especially for problems requiring searching or computing specific values for small cases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.