Reproduce “Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Soichiro Kumano, Hiroshi Kera, Toshihiko Yamasaki
Adversarially pretrained single-layer linear transformers can theoretically achieve robust classification on unseen tasks by leveraging clean in-context demonstrations, effectively transferring robustness without requiring new adversarial training or examples for each downstream task. This approach identifies specific parameter configurations that allow the model to prioritize robust features while mitigating vulnerability to adversarial perturbations.
Key insights
- Adversarial pretraining enables the model to adaptively prioritize robust features (class-discriminative) while suppressing the influence of non-robust features (vulnerable to perturbations), leading to universal robustness.
- Universal robustness is bounded; it can fail if the adversarial perturbation scale exceeds a certain threshold or if non-robust features overwhelmingly outnumber robust ones.
- Adversarially pretrained transformers exhibit an accuracy-robustness trade-off and require more in-context demonstrations to achieve clean accuracy comparable to standardly trained models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.