Reproduce “Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hao Wang, Guozhi Wang, Han Xiao, Yufeng Zhou, Yue Pan, Jichao Wang, Ke Xu, Yafei Wen and 3 more

Skill-SD uses an agent’s completed attempts to create task-specific skills for a training-only teacher, while the acting student keeps the plain task prompt. GRPO rewards and teacher token-level guidance jointly train the student. On AppWorld and Sokoban, it outperformed vanilla GRPO; on AppWorld, pass@1 was 64.9% versus 50.9%. Evidence is limited to two benchmark settings.

Key insights

  • Skills are most effective when they condition only the teacher during training, facilitating skill internalization by the student without introducing inference-time dependencies or causing overfitting.
  • Student-owned (on-policy) rollouts are essential for stable self-distillation in multi-turn agent training; teacher-owned (off-policy) rollouts consistently lead to training collapse.
  • Dynamic synchronization of the teacher's parameters from the latest student checkpoint is critical for sustained performance gains, ensuring the distillation signal co-evolves with the student's improving policy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.