Reproduce “Defending Jailbreak Prompts via In-Context Adversarial Game” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yujun Zhou, Yufei Han, Haomin Zhuang, Kehan Guo, Zhenwen Liang, Hongyan Bao, Xiangliang Zhang
An In-Context Adversarial Game (ICAG) framework allows Large Language Models to defend against jailbreak prompts by iteratively refining attack and defense strategies via in-context learning, reducing jailbreak success rates to 0% for GPT-3.5-Turbo on various attacks and demonstrating transferability with only a 2.86% average increase in JSR.
Key insights
- Adversarial training principles, when combined with in-context learning and agent-based systems, can build robust LLM defenses without fine-tuning.
- An iterative adversarial game allows for dynamic and continuous refinement of both jailbreak attack prompts and defense system prompts.
- Defense strategies derived from this in-context game are transferable across different Large Language Models, enhancing broad applicability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.