Reproduce “Unbiased Reward Modeling from Implicit Feedback for LLM Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Wang, Haocheng Yang, Licheng Pan, Lei Shen, Xiaoxi Li, Yinuo Wang, Zhichao Chen, Yuan Lu and 2 more
ImplicitRM trains reward models from implicit user feedback by estimating both response preference and the likelihood of eliciting an action, rather than treating every unmarked response as negative. On PKU-SafeRLHF, with feedback simulated from explicit labels, it achieved an R-squared of 0.7872, versus 0.7540 for LaGAM. This result is limited to that evaluation; unbiasedness requires accurate group probabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.