Reproduce “AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun and 2 more
AutoDAN-Turbo proposes a multi-agent, lifelong learning framework that autonomously discovers and evolves diverse jailbreak strategies for large language models (LLMs) in a black-box setting. The method achieves an average 70.4% to 74.3% higher attack success rate compared to baselines and demonstrates strong effectiveness on GPT-4-1106-turbo with an ASR of up to 88.5%.
Key insights
- Automated, LLM-driven discovery of jailbreak strategies can significantly outperform human-dependent methods in identifying LLM vulnerabilities.
- A lifelong learning paradigm allows for the continuous evolution and combination of jailbreak strategies, leading to more sophisticated and effective attacks.
- Learned jailbreak strategies demonstrate strong transferability across different target LLMs and datasets, enhancing the generalizability of red-teaming efforts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.