Reproduce “Selective Regenerative Decoding: Trajectory-Level Intervention for Inference-Time Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sophia Xiao Pu, Yumo Xu, Sailik Sengupta, Millennium Bismay, Ruixue Lian, James Gung, Yi-an Lai, Arshit Gupta
Selective Regenerative Decoding (SRD) enhances large language model reasoning by introducing segment-level intervention, allowing it to preserve high-quality reasoning prefixes and selectively regenerate degraded suffixes. This approach achieves accuracy comparable to Best-of-N methods with significantly fewer generated tokens, demonstrating improved sample efficiency across various reasoning benchmarks.
Key insights
- Treating reasoning trajectories compositionally, rather than as atomic units, allows for targeted intervention and recovery of valuable partial computations.
- Selective regeneration of degraded suffixes yields a proven sample efficiency gain and a strictly higher expected quality of the best-found trajectory compared to pure rejection sampling.
- The effectiveness of segment-level intervention is dependent on well-calibrated routing thresholds and a reliable reward model for localizing degradation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.