Reproduce “Generative Retrieval Overcomes Limitations of Dense Retrieval but Struggles with Identifier Ambiguity” on OpenResearch

About this paper

Adrian Bracher, Svitlana Vakulenko

Researchers at the Vienna University of Economics and Business empirically demonstrate that Generative Retrieval models overcome the “vector bottleneck” limitations inherent in Dense Retrieval on a synthetic benchmark, but also identify a new failure mode where GR struggles with identifier ambiguity on extended datasets, revealing current decoding and scoring weaknesses.

Key insights

• Generative Retrieval's autoregressive architecture and parametric memory allow it to bypass the geometric constraints that limit Dense Retrieval models, demonstrating superior performance on tasks requiring high-rank combinatoric retrieval.
• Current GR models exhibit a significant weakness in handling identifier ambiguity, struggling to distinguish relevant from lexically similar irrelevant documents when identifiers are not unique to relevant documents.
• The choice of decoding strategy (e.g., classical beam search vs. default accumulation) critically impacts GR performance, with more selective pruning preventing noise from irrelevant ngrams from collapsing recall.

Summary by alphaXiv.