Reproduce “Giving AI Personalities Leads to More Human-Like Reasoning” on OpenResearch

About this paper

Animesh Nighojkar, Bekhzodbek Moydinboyev, My Duong, John Licato

Research from the Advancing Machine and Human Reasoning Lab at the University of South Florida demonstrates that Large Language Models can produce more human-like reasoning distributions when prompted with personality traits. The study shows that open-source LLMs, specifically Mistral and Llama, outperform proprietary GPT models in mimicking the diverse and often 'suboptimal' patterns of human judgment on a novel Natural Language Inference dataset.

Key insights

• Modeling the full distribution of human reasoning, including intuitive or 'suboptimal' responses, is crucial for developing truly human-like AI, rather than just optimizing for correct answers.
• Prompting Large Language Models (LLMs) with specific personality traits (inspired by the Big Five model) is an effective and computationally efficient method for eliciting diverse response distributions that better mimic human reasoning patterns.
• Open-source LLMs can outperform larger, proprietary models in replicating the nuanced distributions of human reasoning, challenging the notion that bigger models are inherently more human-like.

Summary by alphaXiv.