Reproduce “SSR-GRPO: Integrating Supervision and Semantic IDs into Reinforcement Learning for Dense Retrieval in E-commerce” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guangxin Song, Xing Fang, Mingmin Jin, Jing Wang, Bokang Wang, Zhentao Song, Junjie Bai, Jianbo Zhu
Alibaba Group researchers developed SSR-GRPO, a dense retrieval framework for e-commerce that integrates Semantic Identifiers (SIDs) and supervised signals into a Reinforcement Learning model. It replaces computationally heavy and potentially biased LLM-based relevance rewards with a dual-perspective assessment and introduces an advanced hard negative mining strategy, achieving online improvements of 0.61% in User Click-Through Conversion Rate (UCTCVR) and 0.39% in Gross Merchandise Volume (GMV) on TmallAPP.
Key insights
- Integrating discrete Semantic Identifiers (SIDs) provides a robust, unbiased, and computationally efficient signal for guiding reinforcement learning in dense retrieval, outperforming large LLM-based reward models.
- Fine-grained hard negative mining based on hierarchical SID differences, when combined with a DPO-style loss, is crucial for learning nuanced semantic boundaries and achieving higher retrieval accuracy.
- A dynamic weighting mechanism that adaptively balances the high-variance gradients from RL signals and the deterministic gradients from supervised objectives significantly stabilizes training and improves generalization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.