Reproduce “Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks” on OpenResearch

About this paper

Yun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford, Agam Goyal, Vincent V. Frigo, Sijia Yang, Dhavan Shah, Junjie Hu and 1 more

Researchers at the University of Wisconsin-Madison developed a method to align large language model agents with human opinions by leveraging empirically derived human belief networks. This approach significantly enhances the fidelity of LLM-based social simulations by effectively inducing human-like opinion patterns, improving upon methods relying solely on demographic information.

Key insights

• Seeding an LLM agent with a single opinion from an empirically-derived belief network significantly improves its alignment with human opinions on related topics within that same network.
• This alignment is specific to the belief network, indicating that LLMs capture the underlying coherence of human beliefs rather than simply exhibiting general priming or superficial repetition.
• Combining both demographic information and belief network seeding yields optimal alignment, suggesting an additive benefit from both types of input for creating more human-like LLM agents.

Summary by alphaXiv.