Reproduce “Multi-Layer Ranking with Large Language Models for News Source Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenjia Zhang, Lin Gui, Rob Procter, Yulan He
A system recommends trustworthy expert sources for fact-checking news by leveraging Large Language Models (LLMs) within a multi-layer ranking framework. The approach improved expert recommendation accuracy (e.g., ~2% increase in MAP/NDCG@10 with GPT-4) and reduced popularity bias, identifying more diverse relevant sources.
Key insights
- Large Language Models, particularly higher-quality ones like GPT-4, can effectively improve the semantic ranking of expert sources beyond statistical matching methods.
- A multi-layer ranking approach can significantly expand the discovery of relevant sources and reduce popularity bias in recommendations by systematically exploring a broader candidate pool.
- Focusing on primary expert sources (based on their quoted statements) provides a more reliable foundation for fact-checking compared to evaluating news platforms themselves.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.