Reproduce “Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yonggan Fu, Qixuan Yu, Yang Zhang, Shang Wu, Xu Ouyang, David Cox, Yingyan Celine Lin
Researchers at Rice University and MIT-IBM Watson AI Lab identified "Robust Scratch Tickets" (RSTs), sparse subnetworks within randomly initialized neural networks that exhibit inherent adversarial robustness without any training. These RSTs achieve robust accuracy comparable to or exceeding adversarially trained dense models, enabling a novel Random RST Switch (R2S) defense that significantly improves robust accuracy against strong attacks.
Key insights
- Adversarial robustness can be an "inborn" property of specific sparse subnetworks within randomly initialized networks, challenging the notion that extensive adversarial training is indispensable.
- Robust Scratch Tickets (RSTs) represent a "zero-training" variant of the Lottery Ticket Hypothesis, demonstrating that robust properties can be encoded in the network's initial structure and weights.
- Adversarial examples crafted for one RST exhibit poor transferability to other RSTs, forming the basis for a new class of ensemble defenses (Random RST Switch - R2S).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.