Reproduce “BAE: BERT-based Adversarial Examples for Text Classification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Siddhant Garg, Goutham Ramakrishnan

A technique called BAE leverages the contextual understanding of BERT's Masked Language Model to generate adversarial examples for text classification models via context-aware token replacements and insertions. This approach creates attacks that consistently achieve larger drops in target model accuracy while maintaining higher naturalness and semantic similarity compared to existing methods.

Key insights

  • Pre-trained Masked Language Models like BERT can effectively generate contextually appropriate and grammatically sound token perturbations for adversarial text attacks.
  • Combining both word replacement and insertion operations (e.g., BAE-R+I mode) yields stronger adversarial examples compared to relying solely on one type of perturbation.
  • Even robust models like fine-tuned BERT classifiers remain vulnerable to sophisticated, context-aware adversarial examples, achieving significant drops in accuracy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.