Reproduce “Data Unlearning in Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Silas Alberti, Kenan Hasanaliyev, Manav Shah, Stefano Ermon

Researchers at Stanford University developed Subtracted Importance Sampled Scores (SISS), an efficient and stable method for data unlearning in diffusion models that combines the benefits of naive deletion and NegGrad. SISS effectively erases specific training examples while preserving the model's overall generative quality, achieving superior performance on unlearning metrics across diverse datasets and mitigating memorization in Stable Diffusion by nearly 90% on target prompts.

Key insights

  • The naive deletion objective for unlearning can be algebraically decomposed into components corresponding to the full dataset and the unlearning set, which can be unified through importance sampling for computational efficiency.
  • A controlled 'superfactor' combined with gradient clipping allows for explicit boosting of unlearning strength on target data while maintaining overall model quality and stability, interpolating between naive deletion and NegGrad.
  • Effective data unlearning in diffusion models requires a distinct approach from concept unlearning, specifically addressing the precise removal of individual datapoints to comply with privacy and copyright mandates.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.