Reproduce “MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaomi LLM-Core Team, Zongming Qiao, Ziyue Hua, Zirui Ou, Zihao Yue, Zihan Jiang, Zhuo Huang, Zhiyang Chen and 141 more
The authors train coding agents with groupwise grading, comparing attempts at the same task to add quality signals to binary test rewards. Online Groupwise Advantage Redistribution shifts positive credit toward higher-quality passing patches. MiMo-V2.6-Pro’s reported average pass-rate score across three sampled attempts rises from 58.4 to 72.6 on DeepSWE v1.1, but this mixed-task run does not isolate grading or compute effects.
Key insights
- RL gains depend on the surrounding system—not only optimization scale—including task diversity, verifier quality, rollout throughput, and consistency between sampling and training.
- Ranking successful solutions and redistributing learning advantage toward higher-quality ones helped sustain pass-rate gains while moderating rollout-length growth compared with training without GAR.
- The results support iterative improvement through training on designed tasks, but do not establish that a model can autonomously redesign or improve its own training process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.