Reproduce “Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang

D-LoopOPD trains an intermediate loop of a recurrent language model from the deeper loop’s next-token distributions on responses generated by the intermediate loop itself, refreshing the teacher as shared parameters change. On Ouro-2.6B-Thinking, it reaches 71.67 Avg@16 on AIME 2024, versus 65.21 for LoopOPD and 62.08 for vanilla; evidence is limited to the reported Ouro-Thinking evaluations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.