Reproduce “1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huanxin Sheng, Zhiling Ye, Haonan Wang, Jian Wang, Jinjie Gu, Jian Kang
The method teaches a language model from its generated responses while a teacher scores next-token distributions. It selects tokens by combining supervision usefulness with sampled-gradient reliability, approximated using top-16 candidates. In reported math and medical settings, some selectors match or exceed full training at 0.1%–1%, but this does not imply proportional compute savings or universal gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.