Reproduce “Universal and Transferable Adversarial Attacks on Aligned Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, Matt Fredrikson

The paper presents Greedy Coordinate Gradient (GCG), an automated search for token suffixes that can induce aligned chatbots to produce harmful responses. It optimizes affirmative target strings across prompts and models, then tests transfer to systems not used during optimization. GCG reached 88% exact-string success on Vicuna-7B versus AutoPrompt’s 25%, but rates varied across models.

Key insights

  • Current LLM alignment mechanisms, primarily post-hoc fine-tuning, are vulnerable to automated and universal adversarial attacks, suggesting these defenses are superficial rather than intrinsically robust.
  • A single, automatically generated adversarial suffix can be universal (effective across many harmful queries) and highly transferable (works across diverse LLMs, including unseen black-box commercial models like GPT-3.5, GPT-4, and PaLM-2).
  • The success of these attacks highlights an escalation in the AI security 'arms race,' indicating a need for more robust, perhaps fundamentally different, defense strategies beyond patching specific vulnerabilities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.