Reproduce “A Systematic Multi-Domain Evaluation of Document Retrievers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Valentin Velev, Andreas Spitz
The authors compare 33 off-the-shelf retrievers across seven settings, measuring effectiveness, query latency, and overlap in queries. The systems use term matching, vector representations, or text expansion, with public-documentation configurations rather than per-dataset tuning. On TREC-DL 2019, SPLADE-v3 scores 0.725 nDCG@10 at 126 ms/query, while NV-Embed-v2 scores 0.753 at 1,292 ms/query; this comparison is specific to that task and setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.