Reproduce “Theorem Prover as a Judge for Synthetic Data Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joshua Ong Jun Leang, Giwon Hong, Wenda Li, Shay B. Cohen

Researchers at The University of Edinburgh developed a framework, "Theorem Prover as a Judge," that uses a formal theorem prover (Lean) to rigorously verify intermediate reasoning steps in synthetic mathematical data. This approach, combined with iterative autoformalisation and reinforcement learning from prover feedback, improved the mathematical reasoning accuracy of various large language models across multiple benchmarks, achieving competitive performance with significantly less training data.

Key insights

  • Formal verification via a theorem prover provides more robust and accurate feedback for evaluating mathematical reasoning steps than LLM-as-a-Judge, significantly reducing false positives in verification.
  • Iterative refinement of autoformalisation substantially increases the reliability of formal verification, allowing theorem provers to effectively process complex LLM-generated reasoning paths.
  • High-quality, formally verified synthetic data for mathematical reasoning enables significant accuracy gains and data efficiency, allowing smaller datasets to achieve competitive performance compared to models trained on much larger, unverified datasets.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.