Reproduce “Learning Item Representations Directly from Multimodal Features for Effective Recommendation” on OpenResearch

About this paper

Xin Zhou, Xiaoxiong Zhang, Dusit Niyato, Zhiqi Shen

LIRDRec is a multimodal recommender system that learns item representations directly from multimodal features, circumventing item ID embeddings and an observed optimization gradient bias in traditional systems. The model achieved an average of 7.11% improvement in NDCG@20 over leading baselines and demonstrated superior cold-start performance, with further gains when using MLLM/LLM-derived features.

Key insights

• An observed optimization gradient bias in existing MMRSs causes item ID embeddings to be suboptimally learned, under-exploiting the potential of multimodal features.
• Directly learning item representations from multimodal features, without integrating ID embeddings, can overcome this bias and lead to more effective and robust recommendation performance.
• Effective multimodal feature fusion through techniques like 2-D DCT and adaptively weighting modality contributions using a progressive mechanism are crucial for robust multimodal representation learning.

Summary by alphaXiv.