Reproduce “Rating-based Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Devin White, Mingkang Wu, Ellen Novoseller, Vernon J. Lawhern, Nicholas Waytowich, Yongcan Cao
Researchers from the University of Texas, San Antonio and DEVCOM Army Research Laboratory introduce Rating-based Reinforcement Learning (RbRL), a framework that leverages multi-class human ratings of trajectory segments to learn reward functions. This approach enables faster learning and higher agent performance while significantly improving human user experience and feedback efficiency compared to traditional preference-based methods.
Key insights
- Multi-class absolute ratings provide a richer, more informative signal per human query than binary preferences, enabling a "global view" of quality.
- Dynamically determined rating class boundaries help the reward model adapt to the evolving performance of the agent and the human's labeling patterns.
- RbRL offers a more natural, intuitive, and less mentally demanding way for humans to provide feedback, enhancing user experience and increasing data input efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.