Reproduce “Mitigating Reward Hacking in Realistic Scenarios Through Representation Engineering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Austin Meek, Kyle Cox, Matthew Nguyen, Austin J. Brockmeier, Steven Basart
The authors test inference-time activation steering to reduce reward hacking in coding agents. They derive an internal direction associated with hacking and cap unusually high activation along it, while comparing this with uniform activation addition. On Qwen3.5-27B, optimized addition yields a 0% hack rate and 82% solve rate, versus 27% and 75% unsteered, on one coding testbed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.