Reproduce “Enhancing Adversarial Attacks through Chain of Thought” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingbo Su
This research introduces CoT-GCG, an adversarial attack that integrates Chain of Thought (CoT) prompting with gradient-based optimization to enhance the effectiveness of attacks against aligned LLMs. The method generates universal suffixes that induce models to produce harmful content via step-by-step reasoning, demonstrating increased attack success rates compared to existing methods, especially when evaluated by the rigorous Llama Guard system.
Key insights
- Chain of Thought mechanisms, typically used to improve LLM reasoning, can be leveraged to enhance the effectiveness of adversarial attacks.
- Gradient-based optimization of CoT triggers leads to more potent and transferable attacks compared to non-gradient CoT methods or standard affirmative-target attacks.
- LLM alignment processes exhibit uneven effectiveness across different harmful content categories, leading to varying vulnerabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.