Reproduce “Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh Murthy, Tian Lan, Lei Li and 8 more
This work introduces Diversity Empowers Intelligence (DEI), a meta-policy framework that integrates diverse software engineering (SWE) agents using a Large Language Model (LLM) as a code review committee. DEIBASE, its implementation, achieved a 55% issue resolution rate on SWE-Bench Lite, significantly outperforming individual agents and setting a new benchmark.
Key insights
- A substantial and quantifiable diversity exists among SWE agents' problem-solving capabilities, with different agents resolving distinct sets of issues.
- Inter-agent diversity (combining different agent designs) offers greater potential for collective intelligence than intra-agent diversity (multiple runs of the same agent).
- LLMs can effectively function as sophisticated evaluators and arbiters in a 'code review committee' role, providing critical analysis and selection of code patches.
- Requiring LLMs to generate detailed explanations before scoring significantly improves the accuracy and reasoning quality of their patch evaluations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.