Reproduce “Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao and 8 more
The authors introduce rank-surprisal ratio (RSR) to select teacher-generated reasoning trajectories suited to each student before fine-tuning. It combines the student’s relative ranking of observed tokens with their surprisal, favoring informative but not wholly unfamiliar examples. Across five students and 11 teachers, RSR reaches average absolute Spearman correlation of 0.86, versus 0.65 for rule-based quality. Evidence centers on mathematical reasoning.
Key insights
- The effectiveness of reasoning trajectories in knowledge distillation is highly student-specific, with stronger teachers not reliably guaranteeing better student outcomes.
- Optimal learning for student LLMs occurs when reasoning trajectories exhibit both 'absolute unfamiliarity' (high surprisal/low absolute probability) and 'relative familiarity' (low rank among plausible tokens).
- Existing likelihood-based metrics often prioritize familiar data over truly informative content, whereas RSR's design explicitly targets this balance inspired by the 'zone of proximal development' concept.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.