Reproduce “RosePO: Aligning LLM-based Recommenders with Human Values” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiayi Liao, Xiangnan He, Ruobing Xie, Jiancan Wu, Yancheng Yuan, Xingwu Sun, Zhanhui Kang, Xiang Wang

Researchers from the University of Science and Technology of China and Tencent introduced RosePO, a framework to align LLM-based recommenders with human values through targeted preference data construction and personalized preference optimization. This approach improved recommendation accuracy (HR@1 by up to 8.11% over SFT models) and effectively mitigated biases like semantic hallucination and popularity bias.

Key insights

  • Explicitly modeling comparative preferences through carefully designed rejected sampling strategies significantly enhances both the helpfulness (accuracy) and harmlessness (bias mitigation) of LLM-based recommenders.
  • A personalized, data-adaptive smoothing factor is crucial for robustly applying preference optimization techniques like DPO to recommendation systems, where preference labels are often noisy and uncertain.
  • The framework successfully demonstrates that objectives for helpfulness and harmlessness can be pursued concurrently in LLM-based recommenders using a hybrid sampling approach for preference data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.