Reproduce “Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sewon Kim, Jiwon Kim, Seungwoo Shin, Hyejin Chung, Daeun Moon, Yejin Kwon, Hyunsoo Yoon

Researchers at Yonsei University defined and developed tools to diagnose "Affective Hallucination" in LLMs, where models generate responses fostering illusory emotional connections. Their Direct Preference Optimization (DPO) method, using the AHaPairs dataset, reduced these hallucinations to near-zero without affecting general capabilities, establishing a new framework for emotional safety.

Key insights

  • Affective hallucination is a quantifiable and mitigable safety risk, distinct from traditional factual hallucination, which can be reliably assessed by automated evaluators like GPT-4o.
  • Emotional alignment in LLMs should prioritize "responsible boundary-setting" over unconditional empathy to prevent the inadvertent cultivation of psychologically harmful illusions of connection.
  • Targeted DPO fine-tuning is necessary for robust mitigation of affective hallucination, as few-shot prompting alone is insufficient, and model scaling does not inherently reduce this risk.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.