Reproduce “Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hansi Zeng, Chen Luo, Hamed Zamani
A new generative retrieval framework, PAG (Planning Ahead in Generative Retrieval), improves both retrieval effectiveness and efficiency by guiding autoregressive document ID generation with simultaneously computed document-level relevance scores. It achieved a 15.6% relative increase in MRR@10 on MSMARCO Dev while reducing query latency by 22x and memory usage by 7.7x compared to previous approaches.
Key insights
- Integrating document-level relevance scores as a "planning-ahead" mechanism within beam search effectively reduces premature pruning of relevant document ID prefixes during autoregressive generation.
- Combining set-based (lexical) and sequential (semantic) document identifiers within a unified generative model provides a more robust and comprehensive relevance signal for retrieval.
- Simultaneous decoding offers an efficient way to compute approximate document relevance, which can then guide the more computationally intensive autoregressive decoding process to achieve better results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.