Reproduce “Meet Students Where They Are: Student-Prefix Teacher Adaptation for On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu
SPTA adapts a teacher to continue from partial student answers, then freezes it for ordinary on-policy distillation (OPD). The teacher trains on student prefixes with reward for a correct final answer, not to copy the prefix. In mathematical-reasoning experiments, SPTA-OPD raises average accuracy from 30.20 to 33.23; evidence covers five benchmarks and does not establish correct intermediate reasoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.