Reproduce “Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models” on OpenResearch

About this paper

Xiangwen Wang, Ananth Balashankar, Varun Chandrasekaran

This research establishes a systematic scaling-law framework for jailbreak attacks in large language models, quantifying attack success against attacker compute (FLOPs) across diverse methods, models, and harm types. It reveals that prompt-based iterative rewriting (PAIR) is more compute-efficient than gradient-based methods, and that attack success consistently scales with diminishing returns.

Key insights

• All evaluated jailbreak attacks consistently follow a saturating exponential scaling pattern, showing rapid improvement at lower compute budgets followed by diminishing returns.
• Prompt-based iterative rewriting (PAIR) is significantly more compute-efficient than gradient-based optimization (GCG), achieving higher attack success ceilings due to superior optimization quality in prompt space.
• Vulnerability to jailbreak attacks varies strongly by the type of harmful goal (misinformation being easiest) and across different LLM families, while model size primarily affects the approach rate to saturation rather than the ultimate success ceiling within a family.

Summary by alphaXiv.