Reproduce “Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection” on OpenResearch

About this paper

Yachao Zhao, Bo Wang, Yan Wang, Dongming Zhao, Ruifang He, Yuexian Hou

Researchers at Tianjin University developed a self-reflection framework to investigate both explicit and implicit social biases in large language models. They found that LLMs exhibit a consistent explicit-implicit bias inconsistency, with implicit biases increasing with model scale and capability, while explicit biases decrease and are suppressed by alignment techniques.

Key insights

• LLMs consistently show a pronounced explicit-implicit bias inconsistency, mirroring human cognitive patterns, where explicit bias is low but implicit associations are strong.
• Implicit biases in LLMs tend to intensify as models become more capable and scale up in size and training data, contrasting with explicit bias which decreases.
• Current alignment techniques effectively suppress explicit biases but have limited impact on mitigating deeper, implicit biases in LLMs.

Summary by alphaXiv.