Reproduce “Imperfectly Imperfect: Evaluating Graded Cognitive Diversity in LLM Persona Simulations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sion Weatherhead, Ben Newell, Aaron Belbasis, Flora Salim
The authors evaluate whether profile-conditioned language-model agents reproduce human variation across questionnaires, reasoning tests, and written decisions. Across seven runs of four models, questionnaire distributions were closer to people’s than reasoning-test distributions: every agent in four Claude Sonnet 4.6 runs scored 4/4 on both four-item tests, regardless of assigned reflection level. The evidence is limited to these seven runs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.