Reproduce “Deep Unlearn: Benchmarking Machine Unlearning for Image Classification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xavier F. Cadet, Anastasia Borovykh, Mohammad Malekzadeh, Sara Ahmadi-Abhari, Hamed Haddadi

An empirical study thoroughly benchmarked 18 approximate machine unlearning methods for image classification across various datasets and deep neural network architectures, revealing Masked-Small-Gradients (MSG) and Convolution-Transpose (CT) as top performers in balancing performance retention, privacy against inference attacks, and computational efficiency. The work also identified more reliable baselines and highlighted challenges in consistently unlearning certain data points.

Key insights

  • Traditional baselines like Gradient Ascent (GA) are often inadequate, with Fine-tuning (FT) and Negative Gradient Plus (NG+) offering more reliable comparison points.
  • Masked-Small-Gradients (MSG) and Convolution-Transpose (CT) consistently emerged as top-performing approximate MU methods, effectively balancing performance retention, privacy against inference attacks, and computational efficiency.
  • Certain 'difficult-to-forget' data points exist within the forget set that are consistently identified by membership inference attacks, even after rigorous unlearning or full retraining.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.