Reproduce “Scaling Reasoning Efficiently via Relaxed On-Policy Distillation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jongwoo Ko, Sara Abdali, Young Jin Kim, Tianyi Chen, Pashmina Cameron

The paper presents REOPOLD, which trains a smaller model on its own sampled responses using selectively allocated teacher feedback. It limits extreme negative token credits, retains dense supervision early, and later focuses updates on uncertain student tokens. On AIME-25, the 1.5B student surpasses named baselines at 300 steps with over 12× training-sample efficiency, measured in sequences rather than time or total compute.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.