Reproduce “PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mihir Parmar, Palash Goyal, Xin Liu, Yiwen Song, Mingyang Ling, Chitta Baral, Hamid Palangi, Tomas Pfister
The PLAN-TUNING framework, developed by Google researchers with a collaborator from Arizona State University, enables smaller language models to learn step-by-step planning for complex problem-solving. This post-training approach significantly enhances mathematical reasoning performance, yielding up to an 18.4% increase on MATH and more than doubling AIME performance for some models, by embedding planning capabilities directly into model parameters.
Key insights
- Post-training LLMs to internalize step-by-step planning directly into their parameters yields more robust reasoning capabilities compared to relying solely on inference-time prompting.
- A two-stage filtering process (plan verification and answer verification) is critical for generating high-quality synthetic planning trajectories that ensure both logical coherence and factual correctness.
- Incorporating planning as a core model capability significantly improves generalization to out-of-domain and complex reasoning tasks, indicating a deeper understanding of problem-solving.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.