Reproduce “SoK: Robustness in Large Language Models against Jailbreak Attacks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Feiyue Xu, Hongsheng Hu, Chaoxiang He, Sheng Hang, Hanqing Hu, Xiuming Liu, Yubo Zhao, Zhengyan Zhou and 4 more
Researchers from Shanghai Jiao Tong University and Microsoft systematized knowledge on LLM jailbreak attacks and defenses, introducing comprehensive taxonomies and a multi-dimensional 'Security Cube' evaluation framework. Applying this framework to benchmark various LLMs, attacks, and defenses revealed that newer models like o1-mini and Claude-3.7-Sonnet achieve significantly lower attack success rates (~17% ASR) and that multi-agent judges align best with human assessments for safety.
Key insights
- Newer, reasoning-aligned LLMs demonstrate a qualitative leap in jailbreak robustness, with advanced safety mechanisms reducing average ASRs to near zero for some attacks, though no model is fully immune.
- Attack effectiveness and generality converge, with strategy-based and multi-round approaches showing high transferability across diverse LLM architectures, indicating shared latent failure modes.
- Pre-filter defenses, particularly those leveraging internal model states (e.g., Hidden State Guard), are highly effective and efficient at intercepting malicious prompts, while post-generation fixes are often less efficient and can degrade utility.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.