Reproduce “AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruption” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joonsung Jeon, Woo Jae Kim, Suhyeon Ha, Sooel Son, Sung-eui Yoon

ADVPAINT protects images from diffusion-based inpainting manipulation by crafting adversarial perturbations that directly disrupt self- and cross-attention mechanisms. The method significantly outperforms prior defense strategies, achieving an FID of 303.90 for background inpainting, 148.33 points higher than the next best baseline.

Key insights

  • Directly disrupting the attention mechanisms (both self- and cross-attention) of diffusion models is crucial for effectively interfering with their semantic understanding and prompt alignment during inpainting.
  • A two-stage perturbation strategy that applies distinct adversarial noise to different image regions offers enhanced robustness against the diverse and partial manipulations inherent in inpainting tasks.
  • Traditional latent space-focused adversarial objectives are less effective in inpainting scenarios due to the removal of perturbations within masked areas.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.