Reproduce “Diagnosing On-Policy Self-Distillation for Reasoning Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang Li, Gongle Xue, Yuheng Yuan, Yijia Guo, Shizhe Zhang, Liwen Hu, Lei Ma
OPSD trains a model to match next-token scores from a frozen copy that sees extra context while the student solves math problems. Outcomes vary with rollout length, teacher context, and reasoning mode. For Qwen3-1.7B, Avg@8 rises from 34.0 to 38.1 with a 256-token training cap, but longer caps and other models can degrade; evidence covers math models up to 8B.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.