Reproduce “Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sara Rajaram, R. James Cotton, Fabian H. Sinz

SARA uses contrastive learning to represent preferred and non-preferred trajectory sets, then rewards behavior by its cosine similarity to the preferred-set representation. Unlike pairwise reward fitting, it learns shared patterns across preferred examples. On hopper-medium-replay with 10% label error, its mean normalized return was 83.66 ± 3.5 versus 65.21 ± 22.3 for BT Contrastive across eight seeds; at 40% error, all methods degraded.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.