Reproduce “Enhancing CTR Prediction through Sequential Recommendation Pre-training: Introducing the SRP4CTR Framework” on OpenResearch

About this paper

Ruidong Han, Qianzhong Li, He Jiang, Rui Li, Yurou Zhao, Xiang Li, Wei Lin

The SRP4CTR framework developed by researchers at Meituan enhances Click-Through Rate prediction in industrial recommender systems by integrating sequential recommendation pre-training with a Fine-Grained BERT model to encode rich side information. It achieves this efficiently through novel fine-tuning mechanisms, leading to online A/B test gains of 1.66% in Gross Merchandise Volume and 0.70% in CTR on Meituan's platform.

Key insights

• Pre-training on user behavior sequences with self-supervised learning can effectively capture complex user preferences for downstream CTR prediction.
• Encoding rich side information (beyond just item IDs) during pre-training is crucial for improving CTR prediction accuracy in real-world scenarios.
• Efficient knowledge transfer from a pre-trained user model to an item-specific CTR task can be achieved through specialized, low-overhead attention mechanisms that support folded inference.

Summary by alphaXiv.