Reproduce “Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chengbing Wang, Yang Zhang, Zhicheng Wang, Tianhao Shi, Keqin Bao, Fuli Feng, Tat-Seng Chua
The L2D framework, developed by researchers at the University of Science and Technology of China and the National University of Singapore, introduces latent-space decoding to dramatically accelerate inference in generative Large Language Model (LLM)-based recommender systems. This approach achieves over 10x faster inference compared to existing methods while maintaining or improving recommendation performance, enabling practical deployment of advanced LLM4Rec models.
Key insights
- After generative fine-tuning, LLMs form internal, latent-space representations of recommendation targets that can be directly leveraged for efficient item identification without explicit text generation.
- Decoupling the decoding process from language generation allows for substantial efficiency gains while preserving the powerful generative capabilities and knowledge acquired during the LLM's fine-tuning.
- Different aggregation strategies for item representations (global for robustness in sparse data, local for personalized context in dense data) can adapt L2D effectively across varied recommendation scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.