Reproduce “ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts” on OpenResearch

About this paper

Darpan Aswal, Céline Hudelot

The ConceptGuard framework, developed by researchers at Université Paris-Saclay, introduces neuro-symbolic safety guardrails for Large Language Models by extracting interpretable "jailbreak concepts" from LLM internals using Sparse Autoencoders. This approach enables the creation of a robust and explainable post-hoc defense that achieves high True Positive Rates across diverse attack configurations while preserving the base model's capabilities without fine-tuning.

Key insights

• Sparse Autoencoders can reliably extract human-interpretable concepts from LLM activations that represent jailbreak-related themes, indicating a 'shared activation geometry' for diverse adversarial prompts.
• Concept-guided guardrails, built upon these interpretable SAE features, provide robust defense against a variety of jailbreak attacks while maintaining transparency in their decision-making process.
• Pruning non-semantic features from the SAE output significantly reduces the complexity of the safety guardrail while maintaining high efficacy and improving False Positive Rates (FPRs) by focusing on relevant concepts.

Summary by alphaXiv.