Reproduce “MMAligner: Safeguarding Multimodal Large Language Models through Representation Calibration” on OpenResearch

About this paper

Shenyi Zhang, Keyan Guo, Zihao Wang, Xuebin Li, Lingchen Zhao, Hongxin Hu, Chao Shen, Qian Wang

Researchers at Wuhan University, University at Buffalo, and Xi'an Jiaotong University demonstrated that Multimodal Large Language Models' safety failures for harmful multimodal inputs stem from a representation shift, bypassing existing safety mechanisms. Their method, MMAligner, calibrates these representations, achieving near 100% refusal rates on safety benchmarks with minimal utility degradation.

Key insights

• The core safety mechanisms learned in the text-only modality remain functionally active in MLLMs when processing multimodal inputs, but are often untriggered.
• The primary cause of multimodal safety failures is a 'representation shift,' where unsafe multimodal inputs bypass the operative safety mechanism by clustering outside the established refusal boundary.
• Intervening at a specific Transformer layer by calibrating representations along a text-derived safety direction can effectively reactivate existing safety mechanisms without injecting new safety knowledge.

Summary by alphaXiv.