Reproduce “Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation” on OpenResearch

About this paper

Chunlin Liu, Junnian Chen, Haitong Jiang, Jianyu Zhao, Yingsen Pang, Jingchen Li, Jiabiao He, Youming Lu and 2 more

Researchers introduced UNLINK-VL, a real-world benchmark for evaluating cross-modal knowledge unlearning in Vision-Language Models (VLMs). The study found that unlearning effects do not transfer symmetrically across modalities, with multimodal unlearning transferring more effectively to text-only queries than text-only unlearning transfers to multimodal queries, indicating that intra-modal evaluations can overestimate forgetting.

Key insights

• Cross-modal unlearning exhibits significant asymmetry; multimodal unlearning transfers more effectively to text-only evaluation than text-only unlearning transfers to multimodal evaluation.
• Relying solely on intra-modal evaluation, particularly text-only metrics, can substantially overestimate the actual effectiveness of knowledge unlearning in VLMs.
• Forgotten knowledge can often be recovered through relational reasoning (multi-hop queries) or semantically equivalent questions (rewritten queries), indicating that robust unlearning must prevent such recovery while preserving neighboring knowledge.

Summary by alphaXiv.