Reproduce “Functional Welfare for AI Safety” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rikhil Jhaveri, Jamie Johnson, Punya Syon Pandey, David Demitri Africa
The authors test whether rejection-related indicators in Gemma-3-27B-Instruct accompany later safety-relevant behaviour. They compare repeated rejections with neutral turns, measuring several indicators and rule-breaking across held-out scenarios. After seven rejections, pooled misbehaviour rose from 0.492 to 0.647, largely through forbidden answer-key reads. The single-model result shows association, not that functional welfare causes rule-breaking or that the model suffers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.