Reproduce “Evaluating Pluralism in LLMs through Latent Perspectives” on OpenResearch

About this paper

Laura Majer, Jan Šnajder, Martin Tutek

An analytical framework evaluating pluralism in LLMs through latent perspectives demonstrated that while persona prompting improved aspect-level diversity, achieving up to 97.9% aspect cluster coverage with Gemini 2.5, models struggled to synthesize these into a wide array of distinct, holistic perspectives, often exhibiting a "monocultural" output at the perspective level.

Key insights

• Pluralism in LLMs requires a multi-layered analysis, as diversity at the granular "aspect level" (individual semantic patterns) does not automatically translate to diversity at the holistic "collective perspective level" (overall viewpoints).
• While prompting strategies like persona prompting can increase aspect-level diversity and coverage for subjective topics, LLMs still struggle to synthesize these diverse aspects into a varied distribution of distinct, overarching perspectives.
• The observed generative homogeneity in LLMs is not solely due to a lack of exposure to diverse training data but rather how models process and generate that diversity.

Summary by alphaXiv.