Reproduce “ChatGPT for Conversational Recommendation: Refining Recommendations by Reprompting with Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kyle Dylan Spurlock, Cagla Acun, Esin Saka, Olfa Nasraoui
Researchers at the University of Louisville and Microsoft investigated ChatGPT's ability to function as a conversational movie recommender, demonstrating that iterative reprompting with user feedback significantly improves recommendation relevance. The study also identified and mitigated popularity bias in ChatGPT's suggestions through prompt engineering and temperature adjustments.
Key insights
- Iterative feedback and reprompting significantly improve the relevance and quality of ChatGPT's recommendations, highlighting the value of conversational interaction.
- ChatGPT, when used conversationally with feedback, can achieve recommendation performance on par with traditional supervised models like NMF within a content-based evaluation framework.
- Popularity bias is present in LLM-generated recommendations, but can be reduced by specific prompt engineering and increasing the temperature, though this may lead to a trade-off in traditional accuracy metrics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.