Reproduce “InteraRec: Screenshot Based Recommendations Using Multimodal Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Saketh Reddy Karra, Theja Tulabandhula

InteraRec presents a framework that generates personalized product recommendations by analyzing user browsing session screenshots with Multimodal Large Language Models. This approach consistently enhanced the performance of established session-based recommender models, yielding the most substantial gains in data-limited environments.

Key insights

  • Screenshots provide a direct and highly interpretable visual representation of user interaction, simplifying input processing for recommendations.
  • Integrating Multimodal Large Language Models (MLLMs) to analyze visual browsing data allows for real-time and dynamic personalization of recommendations.
  • The framework offers substantial performance improvements for recommender systems, particularly in scenarios with scarce training data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.