Reproduce “LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals” on OpenResearch

About this paper

Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris and 3 more

The authors test whether LLM agents can predict individuals’ later answers from their own interviews and survey responses, without outcome-specific model training. On held-out General Social Survey questions, interview-grounded agents reached 83% of participants’ two-week retest accuracy, compared with 74% for demographic-only agents. This reliability-relative advantage did not extend to significant differences among agents on incentivized economic-game choices.

Key insights

• Rich individual self-report data (interviews and/or surveys) are a highly effective grounding mechanism for LLM agents, enabling general-purpose simulation across diverse tasks without outcome-specific training.
• Grounding LLM agents in deep individual-level information substantially mitigates accuracy disparities across demographic groups (e.g., race, political ideology) compared to demographic-based or brief persona-based approaches.
• The predictive power of self-reports primarily stems from their informational content, rather than specific linguistic stylings, and even significantly shortened interviews retain strong predictive capability.

Summary by alphaXiv.