Reproduce “A Survey of On-Policy Distillation for Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mingyang Song, Mao Zheng

The authors survey on-policy distillation (OPD), organizing it around student-visited states, supervision information, and policy updates. OPD seeks teacher guidance on student-generated prefixes, but that guidance may be unreliable or difficult to learn. Unrelated worked math context helped in one forward-KL setting, while fixed negative coefficients reproduced one sampled-token result, not showing teachers are generally unnecessary.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.