Reproduce “Social Simulations with Large Language Model Risk Utopian Illusion” on OpenResearch

About this paper

Ning Bian, Xianpei Han, Hongyu Lin, Baolei Wu, Jun Wang

Bian et al. found that social simulations using Large Language Models (LLMs) consistently generate idealized, "Utopian" interactions exhibiting positivity, homogeneity, and specific social cognitive biases not reflective of authentic human behavior. Their analysis reveals these biases stem from LLM alignment processes, which prioritize agreeable responses over realistic social complexity.

Key insights

• LLM-driven social simulations tend to create a "Utopian illusion," characterized by overly polite, homogeneous, and coherent interactions that significantly diverge from the complexity and variability of human social behavior.
• LLM agents consistently exhibit emergent social cognitive biases, including a social role bias (over-representation of high-status occupations), a primacy effect (strong influence of early conversational topics), and a pervasive positivity bias.
• These biases are primarily attributed to alignment-driven optimization processes, such as reinforcement learning from human feedback (RLHF), which implicitly favor emotionally positive and semantically aligned utterances, leading to social desirability bias-like patterns.

Summary by alphaXiv.