Reproduce “Which Successes Make Good References? Action-Sequence Alignment for Multi-Turn On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mufei Li, Nikki Lijing Kuang, Zhuoer Wang, Samson Koelle, Jie Yuan, Manqing Mao, Hong Wang, Yanjun Lin and 4 more
The authors introduce reference-augmented distillation, selecting for each failed rollout a same-task success with the longest ordered action subsequence. The teacher uses this reference during training; the student does not see it at inference. On τ³-bench airline tasks, Qwen3.5-4B Pass⁴ rises from 26.7% with vanilla OPD to 38.3% with LCS. Training conditions differ, and overlap does not prove state agreement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.