Reproduce “Testing Robustness Against Unforeseen Adversaries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Max Kaufmann, Daniel Kang, Yi Sun, Steven Basart, Xuwang Yin, Mantas Mazeika, Akul Arora, Adam Dziedzic and 4 more
Researchers from the Center for AI Safety and UC Berkeley introduced a new framework, ImageNet-UA, and a suite of 18 differentiable non-L_p attacks to evaluate machine learning models' robustness against unforeseen adversaries. Their work establishes the Unforeseen Adversarial Accuracy (UA2) metric, demonstrating that existing L_p robustness measures do not generalize to these new threats and identifying strategies like L_2 adversarial training and data augmentation for improved resilience.
Key insights
- Robustness to unforeseen adversaries is a distinct challenge from L_p robustness and requires dedicated evaluation and defense strategies.
- Advances in standard computer vision (e.g., larger models, self-supervised pretraining, data augmentation) positively correlate with improved unforeseen robustness.
- L_2 adversarial training, multi-attack robustness techniques, and perceptual distance-bounded training are more effective for unforeseen robustness than traditional L_∞ adversarial training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.