Reproduce “DESSERT: An Efficient Algorithm for Vector Set Search with Vector Set Queries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Joshua Engels, Benjamin Coleman, Vihan Lakshman, Anshumali Shrivastava
DESSERT is an efficient approximate algorithm for searching large collections of vector sets with set-based queries, offering 2-5x speedups over optimized baselines like PLAID on real-world benchmarks with competitive recall. This approach addresses the scalability limitations of multi-vector representation search, enabling real-time semantic search applications.
Key insights
- Locality-Sensitive Hashing can be adapted to efficiently estimate set-to-set similarities and provide probabilistic guarantees for approximate vector set search by aggregating collision counts.
- The problem of vector set search is distinct and requires specialized algorithms, as simply adapting single-vector search methods leads to inadequate performance.
- Careful engineering of hash table structures and LSH parameters is crucial for achieving practical memory efficiency and query speed when applying LSH to multi-vector sets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.