Reproduce “Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Licheng Pan, Yongqi Tong, Xin Zhang, Xiaolu Zhang, Jun Zhou, Zhixuan Chu
Researchers from Zhejiang University and Ant Group systematically investigate large language model overrefusal by empirically unveiling "safety decision boundaries" within their representation space, proposing an efficient framework that leverages steering vectors to generate targeted overrefusal data. The work introduces MORBENCH, the first large-scale multilingual benchmark, and demonstrates that data filtered by their framework significantly improves overrefusal mitigation through DPO training compared to random sampling.
Key insights
- Overrefusal is strongly linked to a model's "safety decision boundary" in its internal representation space, where benign prompts are misclassified due to subtle nuances.
- LLMs exhibit significant multilingual variability in their safety decisions, with models being more prone to overrefusal in certain languages.
- Targeting prompts near the safety decision boundary is more effective for mitigating overrefusal than using randomly sampled data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.