Reproduce “Reasoning over Precedents Alongside Statutes: Case-Augmented Deliberative Alignment for LLM Safety” on OpenResearch

About this paper

Can Jin, Rui Wu, Tong Che, Qixin Zhang, Hongwu Peng, Jiahui Zhao, Zhenting Wang, Wenqi Wei and 5 more

Researchers from Rutgers University and collaborators developed Case-Augmented Deliberative Alignment (CADA), a reinforcement learning framework that trains Large Language Models to reason over illustrative safety cases instead of extensive rule sets. CADA enhances model harmlessness and robustness against jailbreak attacks while preserving helpfulness and general utility, achieving superior safety performance across benchmarks with minimal training data.

Key insights

• Case-augmented reasoning consistently outperforms rule-based reasoning in terms of harmlessness, robustness, and helpfulness preservation for LLM safety.
• Explicitly referencing extensive safety codes at inference time inconsistently improves harmlessness and systematically degrades helpfulness, leading to over-refusal.
• Reinforcement learning on self-generated reasoning chains (CADA) is effective for safety alignment, even with limited training data, by allowing models to learn adaptive, context-dependent safety judgments.

Summary by alphaXiv.