Reproduce “Revisiting Adversarial Training at Scale” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zeyu Wang, Xianhang Li, Hongru Zhu, Cihang Xie
UC Santa Cruz researchers developed AdvXL, an efficient two-stage adversarial training framework, to scale adversarial robustness to models with up to one billion parameters and datasets containing over one billion images. This approach established new state-of-the-art robust accuracy records on ImageNet-1K, including an 11.4% improvement in l∞-robust accuracy.
Key insights
- The two-stage training strategy provides a substantial computational speedup (over 5x) without compromising final robustness, making large-scale adversarial training feasible.
- Adversarial robustness consistently improves with increasing model size, data scale, and extended training schedules, demonstrating that model scaling benefits extend to robustness.
- Larger models specifically require larger datasets to fully unlock their robustness potential, mirroring observations in vanilla model training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.