Reproduce “On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yichen Wang, Yufa Zhou, Siqi Zhu, Haozheng Luo, Jingzhi Bao, Zhichao Liu, Zijun Wei
The authors test whether on-policy distillation transfers specialist improvements beyond supervised tasks. A student learns from its own responses with teacher guidance; multi-teacher training routes domains to matching teachers. Narrow math supervision nearly matches teacher scores in-distribution but leaves out-of-distribution performance 3.73% below base. Broader data with a stronger teacher raises scores across five benchmarks, but effects are not isolated.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.