Reproduce “CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas” on OpenResearch

About this paper

Emanuel Tewolde, Xiao Zhang, David Guzman Piedrahita, Vincent Conitzer, Zhijing Jin

Researchers at Carnegie Mellon University, University of Toronto, and Max Planck Institute introduce CoopEval, a benchmark for evaluating large language model (LLM) agents' behavior in social dilemmas and the effectiveness of game-theoretic cooperation mechanisms. The study reveals LLMs' inherent tendency to defect without intervention but demonstrates that mechanisms like Contracting and Mediation can induce up to 80% cooperative outcomes, which further increases under evolutionary dynamics.

Key insights

• Modern LLM agents consistently default to defective actions in social dilemmas without any cooperation mechanisms, a behavior that leads to further welfare deterioration under evolutionary pressures.
• Game-theoretic mechanisms can robustly induce cooperation among LLM agents, particularly Contracting and Mediation, which align self-interest with cooperative outcomes.
• The effectiveness of mechanisms varies significantly across different games and LLM models; simpler mechanisms like Repetition become highly effective under evolutionary dynamics, while Reputation mechanisms are less so, possibly due to LLMs' challenges with complex historical information.

Summary by alphaXiv.