Reproduce “Reward Shaping for User Satisfaction in a REINFORCE Recommender” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Konstantina Christakopoulou, Can Xu, Sai Zhang, Sriraj Badam, Trevor Potter, Daniel Li, Hao Wan, Xinyang Yi and 5 more
A REINFORCE-based recommender system was developed to align user interaction trajectories with underlying user satisfaction by integrating an imputation network for sparse explicit feedback into the reward function. Live A/B testing on a large-scale platform showed an increase of 0.23% in satisfied engagement and a decrease of 0.93% in unsatisfying engagement, alongside positive shifts in direct user ratings.
Key insights
- Addressing class imbalance in sparse satisfaction data, particularly by upweighting negative responses, significantly improves the accuracy of satisfaction imputation.
- Combining user state information (from an RNN) and item-specific features (item ID, creator ID) is effective for predicting user satisfaction.
- A hinge function for combining imputed satisfaction with engagement rewards is more effective in live settings for driving satisfied engagement compared to direct multiplication.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.