Reproduce “Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation” on OpenResearch

About this paper

Ruihong Qiu, Zi Huang, Hongzhi Yin, Zijian Wang

DuoRec adds contrastive learning to a Transformer recommender for next-item prediction. It forms positive pairs using different Dropout masks on one sequence and another sequence with the same next-item target. Across five benchmarks, the authors report improved rankings and more spread-out embedding visualizations. On Beauty, HR@5 is 0.0546 versus 0.0401 for CL4SRec; aggregate results do not establish rare-item gains.

Key insights

• Model-level augmentation through Dropout provides a more semantically consistent and effective way to create positive pairs for discrete sequential data compared to traditional data-level augmentations.
• Leveraging shared target items for supervised positive sampling yields robust, task-aware "hard positives" that significantly enhance the contrastive learning signal.
• The contrastive regularization improves recommendation performance by promoting uniformity and alignment of sequence representations, which implicitly de-degenerates the item embedding space.

Summary by alphaXiv.