Reproduce “LLM-Based Social Simulations Require a Boundary” on OpenResearch

About this paper

Zengqing Wu, Run Peng, Takayuki Ito, Makoto Onizuka, Chuan Xiao

Researchers at the University of Osaka, University of Michigan, Kyoto University, and Nagoya University argue that LLM-based social simulations need clear methodological boundaries due to LLMs' tendency to generate homogeneous outputs, limiting their ability to model diverse social phenomena. They propose a 'boundary-aware' approach, providing a framework and recommendations to enhance the reliability of these simulations for social science research.

Key insights

• The 'average persona' phenomenon in LLMs, driven by training objectives, inherently limits behavioral diversity, causing agents to converge to high-frequency or mainstream behaviors rather than representing the full spectrum of human actions.
• Behavioral heterogeneity, rather than just mean alignment, is a fundamental driver of complex social dynamics and emergent phenomena; simulations lacking this diversity risk producing simplistic or inaccurate collective patterns.
• There is a systematic mismatch between the heterogeneity requirements of social science research questions and the depth of validation practices in current LLM-based simulations, particularly regarding the assessment and reporting of behavioral variance.

Summary by alphaXiv.