Reproduce “ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingyang Yi, Jiazheng Wang, Sida Li
The "ShorterBetter" method introduces a reinforcement learning framework that guides large language models to identify an optimal inference length for efficient reasoning. This approach effectively reduces output token length by 50-80% across diverse reasoning and coding tasks while maintaining or improving accuracy.
Key insights
- An intrinsic optimal reasoning length exists for each problem, dependent on model capacity and difficulty, which balances the need for sufficient analysis with conciseness.
- The Sample Optimal Length (SOL) provides an effective, data-driven approximation of this unobservable optimal length, enabling self-supervised learning for efficient reasoning without external length constraints.
- Reasoning trace analysis reveals that ShorterBetter refines CoT outputs by increasing the proportion of pivotal reasoning steps and reducing non-substantive statements, unnecessary verification, and over-exploration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.