Reproduce “LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation” on OpenResearch

About this paper

Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, Yefeng Zheng

LLMEmb introduces a two-stage framework that leverages large language models to generate high-quality item embeddings, improving sequential recommendations, especially for long-tail items. The method achieves consistent performance gains across various models and datasets by correcting skewed embedding distributions and efficiently integrating semantic and collaborative signals.

Key insights

• Fine-tuning a general LLM with supervised contrastive learning based on item attributes effectively adapts it to generate semantically rich and distinguishing item representations for recommendation.
• Freezing the LLM and using a small, trainable adapter during adaptation to inject collaborative signals is crucial for preserving the LLM's rich semantic information while making embeddings suitable for SRS.
• Precomputing and caching LLMEmb-generated item embeddings enables model-agnostic enhancement of any SRS backbone without incurring additional computational burden during real-time inference.

Summary by alphaXiv.