Reproduce “TimelyRAG: Semantic-Temporal Hybrid Retrieval for Time-Critical Question Answering in Overlapping-Evolving Documents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Youngeun Nam, Joeun Kim, Hwanjun Song, Susik Yoon, Jae-Gil Lee, Byung Suk Lee

TimelyRAG adds a temporal-compatibility reranking stage to existing retrieval-augmented generation systems, distinguishing when a document was available from when its rules applied. On the 12,000-example TimelyQABench benchmark, it improves retrieval across evolving legal, university, company-policy, and terms-of-service documents, with gains of up to 28.6% in nDCG@10 and measurable improvements in generated-answer preference at roughly 2–3 ms of reranking overhead per query.

Key insights

  • Temporal validity should be modeled during ranking rather than handled only through recency filters or query rewriting; the relevant document is the one whose clause applies to the query context.
  • Explicit event-time activation and date-granularity modeling are important: removing either reduces University-set nDCG@10 from 0.246 to about 0.205–0.206 for BM25.
  • The method is retriever-agnostic and preserves static-domain behavior, while larger candidate pools help mainly until the correct version is likely to be included.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.