Reproduce “Unsegment Anything by Simulating Deformation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahao Lu, Xingyi Yang, Xinchao Wang
This paper introduces the "Anything Unsegmentable" task and proposes Unsegment Anything by Simulating Deformation (UAD), a two-stage adversarial attack method designed to make images resistant to prompt-based segmentation models. UAD effectively generates adversarial perturbations that are highly transferable across various models and prompt-agnostic, demonstrating superior performance in disrupting segmentation masks compared to existing methods, with an mIoU of 51.53 on white-box SAM-B.
Key insights
- Adversarial perturbations optimized for a specific prompt (e.g., points, boxes) do not generalize well across different prompt types or unseen segmentation models.
- Targeted feature disruptions, where adversarial features are pushed towards a 'natural' but structurally altered image, exhibit superior transferability compared to untargeted feature disruptions for foundation segmentation models.
- The 'Anything Unsegmentable' task is a critical new research direction for proactively addressing ethical and legal challenges posed by powerful promptable AI models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.