Reproduce “Flash-GMM: A Memory-Efficient Kernel for Scalable Soft Clustering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gal Bloch, Ariel Gera, Matan Orbach, Ohad Eytan, Assaf Toledo
IBM Research developed Flash-GMM, a memory-efficient GPU kernel for Gaussian Mixture Models that enables training on datasets up to 100 times larger than previous GPU implementations. This advancement facilitates the use of GMMs as a scalable coarse quantizer in Approximate Nearest Neighbor (ANN) search, yielding improved recall-computation trade-offs through a novel soft multi-assignment strategy.
Key insights
- An IO-aware tiling strategy, inspired by methods like FlashAttention, can effectively overcome memory and bandwidth limitations in other N x K matrix-dependent algorithms like GMMs, making them scalable to very large datasets.
- Leveraging the inherent soft assignments (responsibilities) of GMMs for a multi-assignment strategy in ANN search significantly improves recall-computation trade-offs compared to traditional hard assignments.
- The principled selection of multiple assignments based on GMM responsibilities is crucial for performance gains, outperforming simple hard top-2 assignments based on Euclidean distance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.