Reproduce “EmbSum: Leveraging the Summarization Capabilities of Large Language Models for Content-Based Recommendations” on OpenResearch

About this paper

Chiyu Zhang, Yifei Sun, Minghao Wu, Jun Chen, Jie Lei, Muhammad Abdul-Mageed, Rong Jin, Angli Liu and 4 more

EmbSum, developed by researchers from Meta AI and academic institutions, introduces a content-based recommendation framework that leverages large language model-supervised summarization and poly-attention to efficiently process long user histories. The system achieves state-of-the-art recommendation accuracy on the MIND and Goodreads datasets with fewer parameters than previous methods, while also generating explainable user interest summaries.

Key insights

• LLM-generated summaries serve as an effective supervisory signal to distill complex user engagement histories into meaningful and accurate user interest representations.
• Poly-attention mechanisms, generating multiple embeddings for users and content, are crucial for capturing multifaceted interests and improving interaction modeling.
• Designing a recommendation framework that enables offline pre-computation of user and item embeddings is vital for achieving high scalability and low latency in real-world large-scale systems.

Summary by alphaXiv.