Reproduce “Learning Dynamics of On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guan Zhe Hong, Sumeet Ramesh Motwani, Liang Chen, Yangchen Pan, Junchi Yu, Jialin Yu, Philip Torr, Yi Ren
On-policy distillation trains a language-model student on its own continuations using a teacher’s token-level feedback. The authors trace how feedback changes parameters and predictions, and how those changes redirect training prefixes. A larger teacher transfers less than a smaller RL-trained sibling: the same 1.5B student reaches AIME accuracy of 0.31 versus 0.45. Findings are limited to models and settings tested.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.