Reproduce “Reasoning Can Be Restored by Correcting a Few Decision Tokens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Changshuo Shen, Leheng Sheng, Yuxin Chen, An Zhang, Xiang Wang
Researchers at the University of Science and Technology of China and the National University of Singapore discovered that reasoning discrepancies between standard and reasoning-enhanced language models are concentrated in a small number of early, planning-related decision tokens. They introduced a disagreement-guided token intervention method, enabling a smaller base model to recover or exceed the reasoning performance of a larger reasoning model with minimal intervention, achieving an average Pass@8 of 80.0 compared to 36.0 for the base model on math benchmarks.
Key insights
- Reasoning performance differences are highly sparse and concentrated in a few 'decision tokens,' primarily occurring at early stages of generation (e.g., 8% of tokens account for most disagreement).
- These critical disagreement spikes coincide with moments of high intrinsic uncertainty in the base model and are strongly enriched for planning-related tokens, suggesting base models struggle most at strategic decision points.
- Aggregated token-level disagreement scores are effective predictors of ultimate reasoning failures in base models, outperforming base entropy in diagnostic utility.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.