Reproduce “SciNUP: Natural Language User Interest Profiles for Scientific Literature Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mariam Arustashvili, Krisztian Balog
SciNUP introduces a large-scale, synthetic dataset for evaluating natural language (NL) user profile-based scientific literature recommendation, addressing a critical scarcity of public resources in this domain. The work demonstrates that an ensemble model, combining sparse, dense, and LLM-based approaches, achieves superior recommendation performance with an NDCG@10 of 0.4481, significantly outperforming individual components.
Key insights
- Broader natural language user profiles consistently lead to degraded recommendation performance across sparse, dense, and LLM-based methods.
- Different retrieval and reranking models exhibit complementary strengths, indicating that no single approach is universally optimal across all users.
- Hybrid ensemble approaches, combining diverse retrieval paradigms, significantly outperform individual models by effectively leveraging their complementary relevance signals.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.