Reproduce “SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zheng Li, Qingxiu Dong, Jingyuan Ma, Di Zhang, Kai Jia, Zhifang Sui
The SelfBudgeter framework enables large language models to autonomously estimate and adhere to minimal token budgets for complex reasoning tasks, leading to an average 61% reduction in response length on mathematical benchmarks. This approach enhances efficiency and provides precise control over output while maintaining or improving reasoning accuracy.
Key insights
- LLMs can be trained to autonomously estimate the minimal token budget required for a correct answer, adapting their computational effort to problem complexity.
- Significant token length compression (up to 61%) is achievable by eliminating conversational filler, often without altering core reasoning logic and sometimes improving accuracy by reducing distractions.
- A dynamic linear schedule for the tolerance coefficient ('alpha') in the reward function is crucial for stable training, preventing issues like 'reward hacking' and premature budget collapse.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.