Reproduce “Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models” on OpenResearch

About this paper

Naman D Singh, Francesco Croce, Matthias Hein

This research systematically investigates adversarial training on ImageNet with modern architectures like ConvNeXt and ViT, demonstrating that replacing the standard patch stem with a convolutional stem (ConvStem) consistently improves both -robustness and generalization to unseen - and -attacks. The study achieved new state-of-the-art -robust accuracy for ConvNeXt+ConvStem models, exhibiting, for example, a 5.8% improvement over prior ConvNeXt-T benchmarks.

Key insights

• Replacing the standard PatchStem with a ConvStem consistently and robustly improves adversarial robustness, clean accuracy, and crucially, enhances generalization to unseen - and -attacks across various modern architectures.
• Contrary to some prior assumptions, strong clean pre-training, heavy data augmentation (e.g., RandAugment, CutMix, MixUp), and extended training durations are highly beneficial for achieving state-of-the-art adversarial robustness on ImageNet.
• Increasing image resolution at test time can consistently improve both clean and -robust accuracy for adversarially trained models, particularly for standard perturbation radii, challenging the intuition that higher resolution might strengthen attacks.

Summary by alphaXiv.