Reproduce “Targeted Unlearning with Single Layer Unlearning Gradient” on OpenResearch

About this paper

Zikui Cai, Yaoteng Tan, M. Salman Asif

The paper introduces Single Layer Unlearning Gradient (SLUG), an efficient method for targeted machine unlearning in foundation models like CLIP, Stable Diffusion, and LLaVA. SLUG removes specific information by modifying only a single, critical layer, demonstrating high unlearning effectiveness (e.g., 0.0% Forget Accuracy for CLIP identities) with minimal utility degradation and significantly reduced computational costs (e.g., 39 seconds for Stable Diffusion unlearning).

Key insights

• Targeted unlearning can be achieved effectively and efficiently by modifying only a single, strategically identified layer in a deep neural network.
• Combining layer importance and gradient alignment provides a robust method to select the most impactful layer for unlearning while minimizing side effects on retained knowledge.
• Linearizing the unlearning trajectory to a single-step gradient update significantly reduces computational complexity without sacrificing unlearning effectiveness or utility preservation.

Summary by alphaXiv.