Reproduce “Enhancing Mathematical Reasoning in LLMs with Background Operators” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiajun Chen, Yik-Cheung Tam
Researchers at NYU Shanghai enhance mathematical reasoning in LLMs by training them to generate verifiable Prolog code, utilizing a set of 54 standardized background mathematical operators. Their cross-validated self-training algorithm on the MATH-Prolog corpus achieved an 84.8% accuracy on competition-level problems, showing improved solution computability and diversity.
Key insights
- Prolog's declarative nature is particularly well-suited for expressing complex logical and constraint-based mathematical problems, enabling fully computable and verifiable inference steps for LLMs.
- Standardized 'background mathematical predicates' provide a structured, interpretable framework that guides LLMs in generating consistent and auditable logical programs.
- Cross-validated self-training is an effective method for overcoming the data scarcity in symbolic reasoning, allowing LLMs to autonomously discover and integrate new, diverse, and correct symbolic solutions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.