Reproduce “Real World Evaluation of Approaches to Research Paper Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siddharth Dinesh
This study evaluates various baseline approaches for research paper recommendation, specifically for related article search features in digital libraries, leveraging real-world user click-through rate (CTR) data from the Mr. DLib platform. It quantifies the performance of content-based, random, and stereotype recommenders, finding that Lucene's MoreLikeThis often leads in CTR, with keyphrase-based methods showing competitive performance for English documents with abstracts.
Key insights
- Recommendation performance is significantly influenced by document characteristics (e.g., language, availability of abstracts) and the user context, highlighting the need for context-aware system design.
- Even simple approaches like Stereotype recommendations can achieve competitive CTRs, suggesting their utility for certain contexts or as strong baselines.
- The Keyphrase approach shows refined content-based similarity, outperforming Lucene MLT for English documents with abstracts, but its performance is highly sensitive to the presence of abstracts and keyphrase generation parameters.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.