Reproduce “Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoran Dai, Jiawen Wang, Ruo Yang, Manali Sharma, Zhonghao Liao, Yuan Hong, Binghui Wang
A backdoor attack framework for Text-to-Image diffusion models was developed by researchers from Illinois Institute of Technology and Samsung, demonstrating a method that is practical, generalizable across various models, and robust against existing defenses. The approach achieves high attack success rates, often exceeding 90%, while generating linguistically natural triggers that evade perplexity-based detection and showing limited susceptibility to state-of-the-art defenses like T2IShield.
Key insights
- T2I DMs are vulnerable to subtle, semantically natural triggers, which can be effectively generated using large language models and integrated by leveraging the shared CLIP latent space.
- Bridging the semantic gap between poisoned prompts and arbitrary target images is a key enabler for practical and generalizable backdoor attacks on T2I DMs.
- Current state-of-the-art defenses against T2I DM backdoors are largely ineffective against attacks employing natural language triggers and semantic-space perturbations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.