Reproduce “Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuanpu Cao, Saket Sathe, Hanyu Wang, Ziyi Yin, Fenglong Ma, Jinghui Chen

Progress-Aware Distillation (ProD) trains small language model agents to reduce repeated, unproductive actions while learning from successful teacher trajectories. It compares student and teacher stagnation to adjust training pressure while retaining supervised fine-tuning. Across eight models, ProD generally lowers measured stagnation and improves accuracy; on HotpotQA, Qwen3-4B reaches 70.8%, versus 66.6% with SFT and GPT-4.1’s 71.0%, in tested settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.