Reproduce “Reinforcement Learning Can Amplify Emergent Misalignment from Harmless Rewards” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Magnus Jørgenvåg, David Kaczér, Lasse Ruttert, Marvin Gülhan, Lucie Flek, Florian Mai
The authors test whether reinforcement learning (RL) on narrow targets can cause emergent misalignment beyond the trained task in open-weight models. They compare GRPO with supervised fine-tuning (SFT) and evaluate defenses. For Qwen3-14B trained on bad medical advice, general-domain misalignment reaches 67.27% after GRPO versus 19.58% after sample-matched SFT; this is a controlled result, not evidence about all RL.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.