Reproduce “Rethinking Model Ensemble in Transfer-based Adversarial Attacks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu

A new framework, Common Weakness Attack (CWA), enhances the transferability of black-box adversarial examples by optimizing for "common weakness" properties within model ensembles. This approach achieves up to 30% higher attack success rates against state-of-the-art adversarially trained models and effectively induces misbehaviors in large vision-language models such as Google's Bard.

Key insights

  • Adversarial transferability is correlated with two properties of the loss landscape: the flatness (quantified by the Frobenius norm of the Hessian) and the closeness of the adversarial example to the local optima of individual models.
  • A "common weakness" in model ensembles can be defined as an adversarial example that resides in a flat loss landscape and is simultaneously close to the local optima of the ensemble's training models.
  • Maximizing the cosine similarity between the gradients of different models in an ensemble encourages the adversarial example to be closer to their shared local optima, which is crucial for enhanced transferability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.