Reproduce “Reinforcing What the Student Has Learned: Teacher-Guided On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bozhen Fang, Linhao Zhong, Xuehao Wang, Huanyi Zheng, Hao Chen, Chunhua Shen
SD-OPD trains a student using its own changes from its pretrained base to set token-update strength, while teacher agreement selects direction; the student signal is refreshed each round. In a six-task mathematical-reasoning evaluation, it reaches 76.11% average pass@1, versus 65.74% for on-policy distillation and 73.82% for its teacher. Evidence is limited to tested Qwen3 students and math benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.