Reproduce “On Jailbreaking Quantized Language Models Through Fault Injection Attacks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Noureldin Zahran, Ahmad Tahmasivand, Ihsen Alouani, Khaled Khasawneh, Mohammed E. Fouda
Research explored hardware-level bit-flip attacks to jailbreak safety-aligned large language models, specifically examining their efficacy across FP16, FP8, INT8, and INT4 quantization schemes. Findings show that FP8 quantization provides robust protection, while INT4 unexpectedly offers minimal resilience, akin to unquantized FP16 models.
Key insights
- Quantization schemes exhibit differential resilience to bit-flip jailbreaking: FP8 significantly increases robustness, while INT4 offers surprisingly little protection.
- Highly constrained bit-level attacks can achieve Attack Success Rates comparable to word-level attacks on FP16 models, demonstrating the potency of minimal, targeted hardware perturbations.
- Vulnerability patterns vary by quantization: FP16/INT4 attacks often target Attention `v_proj` modules, while FP8/INT8 attacks focus on MLP `down_proj` components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.