Reproduce “Hierarchical Retrieval at Scale: Bridging Transparency and Efficiency” on OpenResearch

About this paper

Shubham Gupta, Zichao Li, Tianyi Chen, Cem Subakan, Siva Reddy, Perouz Taslakian, Valentina Zantedeschi

Researchers at ServiceNow Research, McGill University, Université Laval, and Mila introduced ReTreever, a hierarchical retrieval method that learns an optimized tree structure for efficient and transparent information retrieval across text, image, and speech. ReTreever achieved up to 160 times faster query processing than LLM-based hierarchical methods with an average 30% nDCG@10 performance improvement, while providing interpretable, multi-granularity representations.

Key insights

• An efficiently learned, retrieval-optimized tree structure, operating on embeddings without LLM calls, can simultaneously enhance retrieval performance, offer interpretability, and provide flexible multi-granularity representations at scale.
• The learned hierarchy in ReTreever naturally induces semantically coherent groupings of documents across diverse modalities (text, image, speech), providing an inspectable view into data organization and the retrieval process.
• Stochastic depth training improves the quality and utility of intermediate (coarse) representations within the hierarchical structure, enabling effective trade-offs between computational budget and retrieval accuracy.

Summary by alphaXiv.