Reproduce “Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, Zhangyang Wang
Researchers from Texas A&M University, MIT-IBM Watson AI Lab, and Microsoft demonstrate that combining self-supervised pretraining with adversarial training enhances neural network robustness against adversarial attacks and common corruptions. This approach, particularly an ensemble of self-supervised tasks, achieved 54.5% robust accuracy on CIFAR-10 while maintaining competitive standard accuracy.
Key insights
- Adversarial pretraining significantly boosts robustness when followed by adversarial fine-tuning and accelerates convergence.
- Different self-supervised tasks lead to models with diverse adversarial vulnerabilities, which can be leveraged for better overall robustness.
- An ensemble of diverse self-supervised tasks improves both adversarial robustness and robustness against common corruptions by leveraging complementary strengths.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.