Reproduce “Composite Adversarial Attacks” on OpenResearch

About this paper

Xiaofeng Mao, Yuefeng Chen, Shuhui Wang, Hang Su, Yuan He, Hui Xue

Composite Adversarial Attack (CAA) frames adversarial evaluation as a multi-objective search over attack types, ordering, and hyperparameters. By applying attacks sequentially rather than only comparing their independent outputs, CAA generally lowers robust accuracy than individual attacks or simple ensembles, while using about 800 gradient evaluations versus 4,850 for AutoAttack in a reported CIFAR-10 setting; on unrestricted attacks, it reduces robust accuracy to 7.9% and 4.0% for two defenses.

Key insights

• Serially composing complementary attacks is stronger than choosing the best individual attack or combining independently generated outputs, particularly when defenses are specialized to isolated attack types.
• Automated attack-policy search can improve evaluation efficiency: in one CIFAR-10 l∞ experiment, CAA used roughly one-sixth as many gradient evaluations as AutoAttack while achieving lower robust accuracy.
• Unrestricted robustness is especially vulnerable to composition: defenses retaining roughly 95–100% robust accuracy against individual operations fell to 7.9% and 4.0% under CAA.

Summary by alphaXiv.