Reproduce “SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection” on OpenResearch

About this paper

Maithili Joshi, Palash Nandi, Tanmoy Chakraborty

Researchers at the Indian Institute of Technology Delhi introduced SABER, a white-box jailbreak method that bypasses safety alignment in Large Language Models by introducing a strategically scaled cross-layer residual connection during the forward pass. This architectural intervention achieves substantially higher attack success rates on prominent benchmarks compared to existing methods, while revealing that safety mechanisms are localized within middle-to-late model layers.

Key insights

• Safety mechanisms in LLMs are predominantly concentrated in the middle-to-late layers of the transformer architecture, making them susceptible to targeted internal bypasses.
• Introducing a carefully scaled alternative information pathway (residual connection) is more effective for jailbreaking and preserves model coherence better than simply skipping layers, which can lead to increased hallucinations.
• There is a trade-off between attack success and the preservation of reasoning capabilities; higher jailbreaking effectiveness often correlates with some degradation in tasks like MMLU.

Summary by alphaXiv.