Reproduce “Recursive Self-Improvement through Multi-Agent Self-Supervision” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Somayeh Sojoudi, Matei Zaharia, Yujin Tang
MASS improves open-ended research agents by having model copies coordinate through task-specific workflows, then fine-tuning their shared model on selected orchestrator and subagent trajectories. After two cycles, it achieves 1.2–1.6 times the base model’s score per output token on four research benchmarks. This does not extend to coding benchmarks, where the reported ratios are 0.96 and 0.94.
Key insights
- Coordination traces can act as supervision beyond final answers: delegation rose from 1 of 110 base-model trajectories to 87 of 110 after two cycles, and code handoffs rose from 1 to 83.
- Training on multi-agent traces may use supervised tokens more efficiently: one comparison reported a 68.3% win rate with 28.82 million tokens versus 64.0% with 42 million for a single-agent control, though the comparison was not fully matched.
- Improvements in solving tasks were accompanied by better evaluator agreement and workflow search, suggesting that task-solving traces can transfer to other model roles.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.