Reproduce “A*-Decoding: Token-Efficient Inference Scaling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Giannis Chatziveroglou
MIT researchers develop A*-decoding, an inference-time strategy that applies the A* search algorithm to language model decoding, using process supervision to guide exploration through partial reasoning trajectories and achieving up to 3x greater token efficiency than existing methods while enabling small models like Qwen-1.7B to surpass o1's performance on MATH500 and Llama-3.1-8B to match the 70x larger Llama-3.1-70B on AIME 2024.
Key insights
- Structured search in decoding can enhance reasoning without relying on brute-force sampling or scale-driven gains
- Process supervision enables effective pruning of faulty reasoning paths during generation rather than after completion
- Token efficiency is a critical but overlooked dimension in inference-time scaling that significantly impacts practical deployment costs
- Small models can achieve competitive performance through better inference-time computation allocation
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.