Reproduce “BOAD: Discovering Hierarchical Software Engineering Agents via Bandit Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Iris Xu, Guangtao Zeng, Zexue He, Charles Jin, Aldo Pareja, Dan Gutfreund, Chuang Gan, Zhang-Wei Hong
BOAD (Bandit Optimization for Agent Design) is a framework that automatically discovers hierarchical multi-agent architectures for Large Language Model-based software engineering agents using multi-armed bandit optimization. The system improves generalization performance, resolving 20.0% of issues on the challenging SWE-BENCH-Live benchmark with a 36B parameter model, surpassing larger state-of-the-art LLMs.
Key insights
- Intelligent architectural design, specifically the automated discovery of hierarchical multi-agent systems, can yield greater performance gains for complex tasks than simply scaling up LLM size.
- Automated, principled discovery of sub-agent roles and orchestration is more effective than brittle manual designs for LLM-based software engineering agents.
- Small, focused teams of two sub-agents demonstrated optimal performance, indicating a balance between specialization and coordination is crucial, outperforming both monolithic and overly complex multi-agent setups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.