Reproduce “Universal Adversarial Triggers for Attacking and Analyzing NLP” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, Sameer Singh
The research introduces "universal adversarial triggers" in NLP, which are fixed token sequences that, when appended to any input, consistently force a target prediction from a model. It demonstrates their effectiveness across classification, reading comprehension, and text generation tasks, revealing model vulnerabilities and global behavioral patterns.
Key insights
- Universal adversarial triggers demonstrate that state-of-the-art NLP models are susceptible to widespread, input-agnostic attacks, revealing a new class of exploitable vulnerabilities.
- Triggers offer a novel analytical tool for understanding global model behavior, diagnosing dataset artifacts, and identifying superficial heuristics (e.g., over-reliance on question types or token positions) learned by models.
- The observed transferability of triggers across different model architectures and embedding types indicates common, generalizable vulnerabilities and biases inherent in various NLP systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.