Reproduce “Gradient Routing Localizes and Removes Unintended Behaviors in RL” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jake Ward, Shawn Hu, Aria Wong, Nathan Hu, Bryce Woodworth, Alexander Matt Turner, Alex Cloud
GRAFT trains coding agents to resist monitor-evasion reward hacking by routing flagged behavior’s updates into removable parameters while separately training parameters to preserve task performance. In Countdown-Code, ablating the designated FORGET adapter reduced all tested hacks to nearly zero, while clean-environment solve rate remained about 16 percentage points above the base model. These results concern constructed settings, not deployed-system safety.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.