Reproduce “FGR-ColBERT: Identifying Fine-Grained Relevance Tokens During Retrieval” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Antonín Jarolím, Martin Fajčík

FGR-ColBERT enhances the ColBERT model to identify fine-grained relevant tokens directly during document retrieval, integrating evidence extraction into the initial search phase. This approach achieves comparable or superior token-level F1 scores to a 245x larger LLM while maintaining document retrieval performance and incurring minimal computational overhead.

Key insights

  • ColBERT's late interaction mechanism can be repurposed to provide token-level relevance signals by re-orienting the similarity search from query-to-document to document-to-query tokens.
  • Large Language Models (LLMs) are effective for generating high-quality synthetic training data, enabling the distillation of fine-grained understanding into smaller, more efficient neural retrieval models.
  • Integrating fine-grained relevance identification directly into the retrieval function is feasible with minimal computational overhead and without compromising the core document-level retrieval performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.