Reproduce “PMRecBench: A Benchmark for Personalization in Recommendation Agents under Multimodal Queries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoyu Zhang, Yougang Lyu, Bin Wu, Sahan Bulathwela, Emine Yilmaz
The authors introduce a benchmark that tests whether recommendation agents meet image-and-text requests while following users’ past preferences. It pairs multimodal queries with real user histories and compares ranking with and without the query. In clothing tests, adding the query improved overall ranking but lowered preference-following scores by 0.046. This finding applies to the tested agents and benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.