Reproduce “ShallowDefend: Harnessing Shallow Safety Alignment for Multi-Turn Defender” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiyao Ma, Xiaodong Liu, Weiwei Yang, Mingqian Feng, Chengquan Guo, Jialin Song, Carolyn Buractaon, Zhe Zhao and 4 more
ShallowDefend trains language models to assess intent, emit a readable comply-or-refuse decision, and then answer, using reinforcement learning on multi-turn conversations. Its self-verified reward checks the decision against known harmful-or-benign labels and requires a reply, but does not assess reply content. On five safety suites, Qwen3-8B reaches 93.4% average SAFE versus MUSE-D’s 85.7%; results cover scripted attacks, not adaptive adversaries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.