Reproduce “Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tae-Young Lee, Juwon Seo, Jong Hwan Ko, Gyeong-Moon Park
Researchers from Korea University, Kyung Hee University, and Sungkyunkwan University developed Anti-Personalized Diffusion Models (APDM), a model-level defense framework designed to prevent the unauthorized personalization of specific subjects in text-to-image diffusion models. This approach demonstrates enhanced robustness against clean data and image transformations, outperforming previous data-poisoning techniques while preserving the model's overall generative quality and ability to personalize non-targeted subjects.
Key insights
- Directly applying existing data-poisoning loss functions to model parameters for anti-personalization leads to conflicting optimization objectives and convergence failures.
- Effective model-level protection requires a specialized loss function (DPO) that explicitly differentiates between suppressing protected content and preserving the model's general generative capabilities.
- Robust defense against adaptive personalization attempts necessitates a dynamic, trajectory-aware optimization approach (L2P) that anticipates and counteracts future fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.