Reproduce “ProofCouncil: An LLM Agent for Solving Open Mathematical Problems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck, Gergely Bérczi, Uri Kreitner, Liam Price, David Holmes
ProofCouncil, an LLM-based agent employing an author-critic architecture with auxiliary computational and advisory tools, was developed to autonomously solve open mathematical problems. This agent achieved the best performance in the FirstProof challenge by providing solutions for 6 out of 10 problems, and independently solved 5 out of 21 researcher-provided problems, notably improving the upper bound for Erdős Problem 539.
Key insights
- An iterative author-critic architecture, mimicking human peer review, substantially improves LLM capabilities for sustained mathematical reasoning and self-correction.
- Augmenting LLM agents with external computational tools (e.g., CAS) and diverse LLM councils provides critical support for verification and strategic exploration.
- Representing agentic workflows as conditional Directed Acyclic Graphs (DAGs) offers a modular and extensible framework for developing complex AI systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.