Reproduce “LLM-Enhanced Linear Autoencoders for Recommendation” on OpenResearch

About this paper

Jaewan Moon, Seongmin Park, Jongwuk Lee

L³AE uses item text to estimate semantic relationships, then learns collaborative item weights from user interactions while regularizing them toward those relationships. Both phases have closed-form optimal solutions for their objectives. On Games, it achieves Recall@20 of 0.2737 versus 0.2482 for AlphaRec. It leads across three datasets in the NV-Embed-v2 comparison, but not over semantic-only LLM-EASE in Figure 1’s tail subgroup.

Key insights

• LLM-derived dense semantic representations effectively capture richer item relationships, outperforming sparse multi-hot encodings in linear autoencoders.
• A principled two-phase, semantic-guided regularization scheme can effectively fuse heterogeneous interaction and semantic knowledge while preserving the computational efficiency and global optimality of linear models.
• Linear models, when enhanced with powerful semantic features from LLMs, can consistently outperform more complex non-linear deep learning models, especially in sparse data environments.

Summary by alphaXiv.