Reproduce “Learning Contextual Retrieval for Robust Conversational Search” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Seunghan Yang, Juntae Lee, Jihwan Bang, Kyuhong Shim, Minsoo Kim, Simyung Chang

Qualcomm AI Research developed ContextualRetriever, an LLM-based system that significantly improves conversational search accuracy across multiple benchmarks while eliminating the additional inference latency typically associated with query rewriting. This approach effectively processes full dialogue history to generate intent-aware retrieval embeddings without explicit rewriting, achieving state-of-the-art performance efficiently.

Key insights

  • LLMs possess the capacity for contextual understanding (e.g., for query rewriting), but this knowledge is not fully utilized in their direct retrieval embeddings from raw conversational input.
  • Selectively extracting retrieval embeddings only from the current query tokens, while using the full dialogue as context, is crucial to leverage history without 'historical interference'.
  • Guiding the model during training with explicit, high-quality rewritten queries enables it to implicitly generate intent-aware representations from original conversational input without requiring runtime rewriting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.