Reproduce “BAMAS: Structuring Budget-Aware Multi-Agent Systems” on OpenResearch

About this paper

Liming Yang, Junyu Luo, Xuanzhe Liu, Yiling Lou, Zhenpeng Chen

BAMAS, a framework from Peking University, UIUC, NTU, and Tsinghua University, structures multi-agent systems by jointly optimizing Large Language Model (LLM) selection and collaboration topology to achieve strong task performance within a specified cost budget. It demonstrates significant cost reductions, up to 86% compared to existing systems, while maintaining comparable or higher accuracy across mathematical reasoning and code generation benchmarks.

Key insights

• Joint optimization of LLM selection and collaboration topology is crucial for effectively balancing cost and performance, outperforming simpler heuristic cost-aware approaches.
• The RL policy learns to adapt multi-agent collaboration patterns to varying budget constraints and task types, becoming risk-averse (favoring simpler topologies) under tighter budgets.
• The 'Planner-Driven' topology, despite its flexibility, was consistently deemed suboptimal by the RL component due to its high cost and potential instability, indicating that more structured collaboration patterns are often more effective within budget constraints.

Summary by alphaXiv.