Reproduce “Steered Not Stirred: Effective Reward Tilting using Surrogate Reward Mixing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alec Sargood, Olivia Gallup, Georg Maierhofer, Noam Ghenassia, Paul Duckworth, Krisztina Sinkovics, Avishek Joey Bose
MARTINI learns a composition of fixed surrogate rewards from a small offline set of oracle scores or rankings, then supplies it to an existing inference-time steerer without querying the oracle during steering. On ImageNet, its mean embedding-space energy distance was 0.0321, versus 0.0498 for the best single surrogate, across seven selected class–prompt pairs, with oracle-based strength selection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.