Reproduce “TEMPO: Scaling Test-time Training for Large Reasoning Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qingyang Zhang, Xinke Kong, Haitao Wu, Qinghua Hu, Minghao Wu, Baosong Yang, Yu Cheng, Yun Luo and 2 more
TEMPO introduces an Expectation-Maximization (EM) inspired framework for Large Reasoning Models that enables continuous self-improvement during inference on unlabeled data. It achieves sustained performance gains of up to 23.5 percentage points in mathematical reasoning and maintains output diversity, outperforming prior test-time training methods.
Key insights
- Prior self-rewarding TTT methods can be understood as incomplete EM procedures, where the lack of an E-step (critic recalibration) causes performance saturation and diversity collapse.
- Periodic recalibration of the critic on a labeled dataset is crucial for providing a stable and informative training signal, preventing reward drift and enabling sustained performance improvement.
- Utilizing a dynamically calibrated critic that assigns continuous, quality-aware scores effectively preserves output diversity among high-quality solutions, enhancing model robustness and exploration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.