Reproduce “Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chao Gong, Kai Chen, Zhipeng Wei, Jingjing Chen, Yu-Gang Jiang
A new method, RECE, efficiently and thoroughly erases undesirable concepts from text-to-image diffusion models, achieving high robustness against red-teaming tools while preserving general generation quality. This approach utilizes an iterative 'derive-erase' loop based on closed-form solutions, modifying only 2.23% of the U-Net's parameters in approximately 3.4 seconds, demonstrating superior safety against adversarial attacks and improved unlearning completeness.
Key insights
- An iterative 'derive-erase' adversarial loop, which continually identifies and removes residual concept representations, is effective for achieving thorough concept erasure and enhancing robustness against red-teaming attacks.
- The inherent bias in large text-to-image models (e.g., Stable Diffusion's bias towards female nudity) is reflected and potentially perpetuated by current erasure methods, indicating a need for fairness-aware unlearning.
- Efficient concept erasure can be achieved by exclusively modifying a small fraction of the model's parameters (specifically, Key and Value projection matrices in cross-attention) through closed-form solutions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.