Reproduce “Order-Only On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guangzeng Han, Yu Wu, Xiaolei Huang
O³PD trains a smaller language model on its own generated prefixes, using a stronger teacher’s top-K token order rather than matching probability values. It preserves candidate order, not probability gaps. Across six Qwen3 pairs on three mathematical-reasoning benchmarks, it raises mean accuracy in all six, but improves only 14 of 18 individual comparisons.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.