Reproduce “Harsher on Male? Evaluating LLMs on Gender-Asymmetric Moral Framing Across Diverse Conflict Scenarios” on OpenResearch

About this paper

Guangzong Si, Dong Wang, Zhenhao Li, Yifan Yu, Panwang Pan, Wentao Zhu

Researchers from the University of Science and Technology of China developed GAMA-Bench to assess whether Large Language Models apply consistent moral standards to negative behaviors based on the actor's gender. Their evaluation across 10 LLMs revealed a systematic male-disadvantaging asymmetry, where male actors received more punitive and blame-centered framing, while female actors received more therapeutic and empathy-oriented responses for identical misconducts.

Key insights

• Gender bias in LLMs manifests subtly in the *linguistic framing* of responses, affecting tone, blame allocation, and recommended actions, even when the underlying behavior is equally condemned.
• The observed male-disadvantaging asymmetry is a pervasive characteristic, consistently found across 10 diverse LLMs, varying model scales, post-training configurations, and reasoning modes.
• Neither increasing model scale, instruction tuning, nor activating extended reasoning capabilities (
• Chain-of-Thought
• ) systematically reduces these gender-asymmetric framing biases; in some cases, they are amplified.

Summary by alphaXiv.