Reproduce “SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiaqi Wu, Chen Chen, Chunyan Hou, Xiaojie Yuan

SafeInt, developed by researchers from Nankai University, provides a defense for Large Language Models against jailbreak attacks by dynamically intervening in their internal representations. This method relocates harmful prompt representations into a "rejection region" within the model, achieving superior defense against various attack types with minimal impact on utility and negligible computational overhead.

Key insights

  • LLM internal representations of jailbreak, unsafe, and safe samples are empirically distinguishable, with high classification accuracy (over 95%) from intermediate layers onwards.
  • Jailbreak representations generated by diverse adversarial methods (e.g., GCG, AutoDAN, DeepInception) exhibit consistent internal patterns, enabling defense generalization across different attack types.
  • Dynamically relocating jailbreak-related internal representations into the LLM's intrinsic 'rejection region' can effectively activate the model's native refusal mechanisms without extensive parameter alteration.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.