Reproduce “PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chang Wu, Junfeng Fang, Houcheng Jiang, Kai Tang, Pengyu Cheng, Xiaoxi Jiang, Guanjun Jiang, Xiang Wang
PolicyAlign converts natural-language safety policies directly into model behavior through policy-conditioned example generation, policy-sensitive filtering, and on-policy distillation from a teacher model. Across general and domain-specific evaluations, it lowers harmful-response rates while retaining benign-query compliance and general capabilities better than several supervised and preference-based alignment baselines.
Key insights
- Selecting examples by policy sensitivity is more effective than random sampling: the ablation improved defense success from 74.12% to 87.60% on WildJailbreak and from 72.00% to 84.20% on Fortress.
- The approach focuses training on behavior gaps rather than simply increasing refusals, helping preserve benign compliance; Qwen2.5-7B reached 88.8% on XSTest versus 81.6% with SFT.
- Policy documents can serve as a rapid adaptation interface for domain safety, but outcomes remain dependent on policy quality, teacher reliability, synthetic data generation, and automated evaluation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.