Reproduce “RoundTable: Investigating Group Decision-Making Mechanism in Multi-Agent Collaboration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Young-Min Cho, Raphael Shu, Nilaksh Das, Tamer Alkhouli, Yi-An Lai, Jason Cai, Monica Sunkara, Yi Zhang and 1 more
Researchers from the University of Pennsylvania and AWS GenAI explored how different group decision-making mechanisms, specifically various voting rules, influence the quality and efficiency of multi-agent collaboration with Large Language Models. Their study, conducted on the RoundTable platform, revealed that preference voting rules generally lead to more stable and higher-quality outcomes in initial rounds, while language-based early stopping methods reduced collaboration rounds by approximately 50% and improved performance.
Key insights
- Preference-based voting rules generally yield higher and more stable initial collaboration outcomes compared to discrete voting rules, especially in tasks requiring iterative refinement.
- As multi-round collaboration progresses, agent messages become longer and more repetitive, with diminishing informational novelty, suggesting inefficiencies in prolonged discussions.
- Metrics derived from communication patterns, such as information distance and dialogue act transitions, can serve as effective signals for identifying optimal early stopping points in agent collaboration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.