Reproduce “Rethinking GRPO for Large Reasoning Model Safety Alignment” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiawei Chen, Yi Zhang, Jiawei Du, Zhonghou Lv, Tairan Chen, Junyuan Shang, Jinchang Hou, Xiao Yang and 4 more

Beacon addresses safety alignment during group-relative policy optimization by adding safe reference supervision when sampled responses provide no relative safety signal and by using different rewards for harmful and benign prompts. On Qwen3-8B, it reports 1.88% average safety error and 80.64% reasoning pass@1, compared with GRPO’s 15.52% and 78.07%. These results cover the paper’s tested benchmarks and setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.