Reproduce “AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shijue Huang, Hongru Wang, Wanjun Zhong, Zhaochen Su, Jiazhan Feng, Bowen Cao, Yi R. Fung
AdaCtrl presents a framework that enables large language models to adapt their reasoning effort based on problem difficulty and user-specified control, achieving an improved balance between reasoning efficiency and accuracy. The system dynamically allocates computational resources, yielding reductions in response length while maintaining or enhancing performance on mathematical reasoning tasks.
Key insights
- Cold-start fine-tuning with diverse reasoning lengths provides a crucial foundation for adaptive budgeting, enabling models to associate specific difficulty tags with corresponding response styles.
- A dedicated difficulty estimation calibration reward, leveraging GRPO rollouts, effectively teaches the model to accurately assess problem difficulty and align its internal assessment with actual performance.
- The integration of explicit user-controlled tags for "Easy" and "Hard" modes offers robust and precise control over reasoning depth, outperforming simple prompting for length constraint.
- The framework supports a self-adaptive mode that automatically adjusts reasoning effort, striking a balance between effectiveness and efficiency without requiring manual input.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.