Reproduce “Formalize Once, Edit the Rest: Efficient Lean-Based Answer Selection for Math Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ji Feng, Zhouxing Shi
A new answer selection pipeline, BASE, developed by researchers at the University of California, Riverside, significantly reduces the computational cost of formal verification for LLM-generated math answers by decreasing autoformalizer calls by approximately 5x for K=8 candidates. The method, which formalizes a base statement once and edits it for other candidates, also improves selection accuracy by 0.7 to 23.3 percentage points across various mathematical reasoning benchmarks.
Key insights
- Formal statements for different candidate answers to a given math problem exhibit substantial shared structure, which can be leveraged for efficiency.
- Reusing a single, type-checked base formalization and editing it for other candidates significantly reduces formalization "noise" and improves the robustness of Lean type-checking.
- A learned model (LEANSCRIBE) can effectively semantically localize answer expressions within a formal statement and generate a reusable transformation function, enabling efficient formalization of diverse answer encodings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.