Reproduce “LLMs Simulate Big Five Personality Traits: Further Evidence” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Aleksandra Sorokovikova, Natalia Fedorova, Sharwin Rezagholi, Ivan P. Yamshchikov

Sorokovikova et al. investigated whether contemporary large language models (LLMs) can simulate Big Five personality traits and how stable these traits are under varying conditions. The study revealed distinct personality profiles for GPT4, Llama2, and Mixtral, finding that prompt variations consistently influenced traits while temperature sensitivity varied by model.

Key insights

  • Each LLM (GPT4, Llama2, Mixtral) demonstrates a distinct and measurable Big Five personality profile when assessed through standardized questionnaires.
  • Minor prompt variations, such as adding "Answer as if you were a person," consistently affect the perceived personality scores across all models.
  • Model stability to temperature changes varies: Llama2 and Mixtral exhibit remarkable stability with minimal score variation, whereas GPT4 shows measurable responsiveness to temperature shifts.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.