Reproduce “Safety Monitors Mostly Catch What the Model Already Refuses” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sripad Karne
The authors evaluate safety monitors on 647 harmful requests rewritten at three explicitness levels, comparing monitor flags with whether Gemma-4-31B-IT answers. Softer versions draw more answers, and all seven monitors have higher recall on refused than answered requests. WildGuard recall is .47 on answered versus .63 on refused requests. Fine-tuning on matched harmful and harmless wording improves held-out recall but raises false positives. Tests are English and single-turn.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.