Reproduce “Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matthieu Zimmer, Xiaotong Ji, Rasul Tutunov, Anthony Bordg, Jun Wang, Haitham Bou Ammar
Huawei Noah’s Ark Lab and academic collaborators introduce Self-Generated Goal-Conditioned MDPs (sG-MDPs) to enhance automated theorem proving by enabling Large Language Models to dynamically generate and pursue subgoals. Their Bourbaki system, based on this framework, establishes a new state-of-the-art by solving 26 problems on the challenging PutnamBench dataset with 7B parameter models at pass@512.
Key insights
- Decomposing complex problems into dynamically generated, verifiable subgoals addresses the sparse reward problem by providing denser, more immediate feedback.
- Integrating Large Language Models with structured search algorithms like Monte Carlo Tree Search, guided by subgoal verification, enables more effective exploration of the proof space.
- Mirroring human problem-solving strategies through subgoal generation can significantly enhance AI's multi-step reasoning capabilities in formal domains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.