Reproduce “Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gijs Kassenaar, Zhao Yang, Vincent François-Lavet
Researchers at Vrije Universiteit Amsterdam developed a method for language models to dynamically choose between three reasoning modes ('NoThink', 'Short', 'Long') based on problem difficulty, directly within the GRPO framework. This adaptive approach reduced the mean response length by 41% on MATH-500 and 76% on GSM8K, while preserving or enhancing accuracy and demonstrating meaningful routing decisions.
Key insights
- A language model can learn to self-route its reasoning process into distinct, semantically meaningful modes entirely within the GRPO framework, eliminating the need for separate router networks or supervised warm-up stages.
- The learned routing mechanism effectively sorts problems by difficulty, adaptively allocating computational budget by directing easier problems to briefer modes and reserving extended reasoning for harder tasks.
- Hard token caps, a shaped reward surface, and a balance term are effective at preventing routing collapse, ensuring the persistence and distinctiveness of multiple reasoning modes during end-to-end reinforcement learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.