Reproduce “ALERT: Zero-shot LLM Jailbreak Detection via Internal Discrepancy Amplification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiao Lin, Philip Li, Zhichen Zeng, Tingwei Li, Tianxin Wei, Xuying Ning, Gaotang Li, Yuzhong Chen and 1 more
Researchers from the University of Illinois Urbana-Champaign and Visa developed ALERT, a framework for zero-shot LLM jailbreak detection that amplifies internal feature discrepancies. This method consistently achieves over 90% accuracy and F1-score, outperforming other baselines by at least 10% on average across diverse LLMs and attack strategies.
Key insights
- Shallow LLM layers (specifically layer 4) exhibit the largest safety-sensitive distributional disparities between benign, harmful, and jailbreak prompts.
- Gating and context features within LLM Feed-Forward Networks provide stronger zero-shot discriminative signals for detecting jailbreaks than standard hidden states due to 'decoupled activation responses'.
- Noisy tokens in jailbreak templates create a significant distributional gap from benign and harmful prototype feature vectors, which can be used to adaptively down-weight them for improved detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.