Reproduce “Spend Teacher Tokens Where They Matter: Success-Referenced On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiang Chen, Futao Su, Kong Wang, Jiayi Chen, TanLin Li
SR-OPD routes student-generated answers before teacher inference in on-policy distillation. It keeps prompts with both successful and failed attempts, uses a successful answer as a student-side reference, and sends one selected failure to the teacher using sustained hidden-state divergence discounted by estimated input cost. In a one-pass Qwen3-4B-Instruct-2507 to Qwen3-1.7B run, Macro Avg was 37.55 versus 37.48 with 3.64% of Vanilla OPD teacher input; intervals include zero, and findings cover three pairs and six math benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.