Reproduce “Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning” on OpenResearch

About this paper

Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, Maarten de Rijke

Researchers from USTC, University of Amsterdam, and University of Technology Sydney introduce ToolRec, a framework that empowers Large Language Models (LLMs) to act as "surrogate users" for recommendations. ToolRec guides LLMs to iteratively explore user preferences by dynamically invoking specialized attribute-oriented tools, outperforming traditional recommenders and most LLM-based baselines on ML-1M and Amazon-Book datasets, while also addressing issues like hallucination and semantic-behavioral misalignment.

Key insights

• The iterative "Thought, Action, Observation" loop with dynamic tool invocation is more effective for recommendations than rigid, pre-determined execution plans.
• Efficient attribute-oriented retrieval tools, using a frozen backbone and fine-tuned attribute encoder, achieve significant parameter efficiency and scalability, making them practical for a growing number of attributes.
• LLM performance in recommendation is highly dependent on its alignment with domain knowledge, performing well in general domains (movies, books) but struggling in niche ones (local businesses like Yelp).

Summary by alphaXiv.