Reproduce “LikeBench: Evaluating Subjective Likability in LLMs for Personalization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Md Awsafur Rahman, Adam Gabrys, Doug Kang, Jingjing Sun, Tian Tan, Ashwin Chandramouli

Researchers from UC Santa Barbara and Amazon introduced LikeBench, a multi-dimensional benchmark for evaluating the subjective likability and adaptive personalization of Large Language Models in dynamic, multi-session interactions. The benchmark demonstrated that GPT-5 achieved the highest overall likability score (3.94/5) and uniquely maintained its performance in complex, extended dialogues, while memory recall alone did not guarantee high user satisfaction.

Key insights

  • Effective LLM personalization requires not just accurate memory recall, but also the skillful and context-sensitive utilization of information to align with user preferences and achieve subjective likability.
  • Current state-of-the-art LLMs struggle to maintain or improve likability over extended, noisy, and complex multi-session interactions, revealing a significant challenge in sustained adaptability.
  • Some likability dimensions, such as 'humor fit' and 'formality matching', are consistently more difficult for LLMs to master, indicating areas for focused development.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.