Reproduce “MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
LLM-Core Xiaomi
MiMo-V2.6 adds an agentic grader to reinforcement learning for coding agents, comparing patches on precision, minimality, side effects, and craftsmanship instead of binary tests. With Groupwise Advantage Redistribution, MiMo-V2.6-Flash sustained pass-rate improvement through step 52 while turns stayed stable. In a bundled run, Pro’s DeepSWE v1.1 avg@3 rose from 58.4 to 72.6. This result does not isolate GAR’s effect.
Key insights
- Reward quality affects both capability and behavior: groupwise grading prevents uncontrolled growth in interaction turns and token usage and favors smaller, more task-aligned patches over evaluation-specific workarounds.
- Diversity in agent harnesses improves generalization beyond the interfaces used during training, with held-out harness performance rising alongside results on the training harnesses.
- For the MoE model, freezing the router avoids expert collapse during RL: router imbalance and inactive experts remain controlled while benchmark performance continues to improve.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.