Reproduce “P$^2$-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sinuo Wang, Jiacheng Lu, Zeyu Zhang, Maojia Song, Rui Sun, Beidi Luan, Yutong Xie, Jing Li and 3 more
P²-MOPD combines frozen specialist task vectors with trainable student weights and learned coefficients, using on-policy distillation to consolidate domain experts. On Qwen3-1.7B, it reaches 90.5% average recovery after 50 steps, exceeding the same-start MOPD control’s 80.9% final recovery after 100 steps. The measure averages normalized score-gap recovery across medicine, law, finance, and instruction following, not raw accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.