Reproduce “Scaling Reasoning Efficiently via Relaxed On-Policy Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jongwoo Ko, Sara Abdali, Young Jin Kim, Tianyi Chen, Pashmina Cameron
The paper presents REOPOLD, which trains a smaller model on its own sampled responses using selectively allocated teacher feedback. It limits extreme negative token credits, retains dense supervision early, and later focuses updates on uncertain student tokens. On AIME-25, the 1.5B student surpasses named baselines at 300 steps with over 12× training-sample efficiency, measured in sequences rather than time or total compute.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.