Reproduce “The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models” on OpenResearch

About this paper

Giuseppe Russo, Debora Nozza, Paul Röttger, Dirk Hovy

Researchers from EPFL and Bocconi University investigate the alignment of Large Language Models (LLMs) with human moral judgments, revealing a "pluralistic moral gap" where LLMs struggle to reflect the diversity of human moral reasoning, especially in ambiguous situations. They introduce Dynamic Moral Profiling (DMP), which improves LLM-human judgment alignment by 64.3% and significantly increases the diversity of moral values expressed by LLMs.

Key insights

• LLMs exhibit strong alignment with human judgments only in high-consensus moral dilemmas, with alignment sharply deteriorating in ambiguous, low-consensus scenarios where they often default to a single dominant judgment.
• A "pluralistic moral gap" exists, as LLMs disproportionately rely on a narrow set of moral values (top 10 values account for 81.6% of mentions) compared to humans (35.2%).
• Dynamic Moral Profiling effectively bridges this gap by guiding LLMs to express a broader diversity of values and better align with the nuanced, diverse nature of human moral judgments, particularly in complex dilemmas.

Summary by alphaXiv.