Reproduce “Collective cooperation without individual fidelity in LLM agents” on OpenResearch

About this paper

Henrique Ferraz de Arruda, Carlos Gracia Lázaro, Alberto Aleta, Yamir Moreno

This study evaluates how well Large Language Model (LLM) agents mimic human behavior in a networked Prisoner's Dilemma, revealing a "macro-micro dissociation" where aggregate cooperation patterns align with human data, but underlying individual behavioral heterogeneity and specific decision-making rules often do not. The research demonstrates that different LLM models yield varied collective dynamics and that incorporating stochastic agents can improve some aspects of human-like temporal fluctuations.

Key insights

• A "macro-micro dissociation" characterizes LLM agent behavior, where robust macro-level cooperation dynamics can be reproduced while failing to capture the full spectrum of individual-level heterogeneity and context-dependent decision rules observed in humans.
• The choice of LLM model profoundly affects the simulated collective dynamics, demonstrating that different models yield distinct cooperation regimes and are not interchangeable behavioral surrogates without empirical validation.
• Introducing a fraction of random agents in hybrid simulations improved agreement with empirical data in some dimensions, such as temporal fluctuation synchrony and individual-level diversity, suggesting that stochasticity may be crucial for more human-like surrogates.

Summary by alphaXiv.