Reproduce “Heterogeneous User Modeling for LLM-based Recommendation” on OpenResearch

About this paper

Honghui Bao, Wenjie Wang, Xinyu Lin, Fengbin Zhu, Teng Sun, Fuli Feng, Tat-Seng Chua

Researchers from National University of Singapore, University of Science and Technology of China, and Shandong University developed Heterogeneous User Modeling (HUM), an LLM-based approach for recommendation that effectively compresses user preferences from noisy, multi-domain behaviors. HUM integrates compression and robustness enhancers to overcome performance imbalances across domains, demonstrating superior accuracy and scalability on diverse Amazon review datasets.

Key insights

• Direct application of LLMs to multi-domain recommendation is insufficient due to their susceptibility to noise and performance imbalances across domains.
• Explicitly guiding LLMs with targeted prompts and dedicated aggregation tokens (like `[USER]`) significantly enhances their ability to filter noise and compress complex, heterogeneous user preferences into robust representations.
• Dynamically adjusting domain-specific loss weights based on empirical risk, coupled with a smoothing strategy, is crucial for achieving balanced and stable performance across diverse recommendation domains.

Summary by alphaXiv.