Reproduce “Learning Human Objectives by Evaluating Hypothetical Behavior” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siddharth Reddy, Anca D. Dragan, Sergey Levine, Shane Legg, Jan Leike
ReQueST uses a learned environment model to generate hypothetical transitions for users to label, rather than querying only agent-produced behavior. It trains a reward model from simulated-user labels and deploys an agent using that reward. In simulated 2D navigation, it classified all unsafe evaluation states as unsafe and avoided the trap, but this does not establish safety in general.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.