Reproduce “Gorgeous: Revisiting the Data Layout for Disk-Resident High-Dimensional Vector Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peiqi Yin, Xiao Yan, Qihui Zhou, Hui Li, Xiaolu Li, Lin Zhang, Meiling Wang, Xin Yao and 1 more
Gorgeous introduces a system that rethinks data layouts for disk-resident high-dimensional vector search by prioritizing graph structure over exact vectors. This approach significantly boosts average query throughput by over 60% compared to Starling and 78% compared to DiskANN, while reducing latency and disk I/Os.
Key insights
- Adjacency lists (graph structure) are accessed more frequently than exact vectors during graph traversal, making their prioritization in memory caching and disk layout crucial for performance.
- Caching smaller adjacency lists effectively utilizes expensive DRAM, as they are accessed often and their size is independent of vector dimensionality, leading to higher cache hit rates and I/O reduction.
- Graph-replicated disk layouts improve data locality for high-dimensional vectors by prefetching neighbor adjacency lists, which is more effective than traditional graph reordering when few full vectors fit into a disk block.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.