Reproduce “Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, Mario Fritz
Researchers from Microsoft, DFKI, and CISPA Helmholtz Center introduce a complex multi-party, multi-issue negotiation game to evaluate Large Language Models' (LLM) interactive capabilities, strategic reasoning, and safety aspects in cooperative, competitive, and adversarial scenarios. The work demonstrates that while state-of-the-art LLMs like GPT-4 can embody various roles and strategies, the benchmark remains challenging, and the presence of less capable or malicious agents significantly impacts collective success and individual scores.
Key insights
- The proposed benchmark is highly challenging; even state-of-the-art models like GPT-4 underperform a perfect negotiator, and smaller models largely fail, indicating significant room for LLM improvement in complex interactive tasks.
- Critical LLM capabilities for multi-agent negotiation include arithmetic reasoning (preventing 'wrong deals'), theory-of-mind (inferring others' preferences), and strategic planning (maintaining consistent, improving proposals).
- LLMs can successfully embody cooperative, greedy, and adversarial roles based on high-level instructions, with malicious agents significantly reducing negotiation success and potentially forming implicit coalitions or targeting specific parties.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.