Reproduce “Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xixiang He, Xingming Li, Baiqi Wu, Qiyao Sun, Xuanyu Ji, Ao Cheng, Qingyong Hu

Multi-Teacher Self-Distillation Policy Optimization (MT-SDPO) integrates capabilities from specialized Large Language Models into a single student by employing answer-verified, per-sample teacher selection. This method reduces the performance disparity across domains by up to 74.7% and enhances accuracy in the student's weakest areas.

Key insights

  • Domain expertise for LLMs is an average-case property; relying solely on domain-matched teachers for distillation is unreliable, as demonstrated by only 52.9% correctness on average for matched teachers.
  • A substantial portion of samples (27.09% in physics) can be correctly answered by teachers from 'unmatched' domains, underscoring the value of per-sample, answer-verified teacher selection.
  • Answer verification is paramount for effective multi-teacher distillation, as using unverified teacher feedback significantly degrades student model performance below baseline.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.