Reproduce “A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, Amit Portnoy, Shay Vargaftik
A technical note provides a detailed comparison of the recently popularized TurboQuant algorithms with the earlier DRIVE/EDEN vector quantization schemes. The analysis identifies TurboQuant$_{mse}$ as a specific instance of the biased EDEN algorithm with a fixed, often suboptimal, scaling parameter, and demonstrates EDEN's consistent and substantial performance advantages across various metrics and bitwidths.
Key insights
- TurboQuant$_{mse}$ is a special case of the EDEN-biased algorithm, deriving from EDEN by fixing its scalar reconstruction scale parameter (S) to a generally suboptimal value (S=1).
- The multi-stage TurboQuant$_{prod}$ algorithm contains multiple suboptimalities, including its initial (b-1)-bit step using S=1 and its 1-bit Quantized Johnson Lindenstrauss (QJL) residual quantizer performing worse than 1-bit unbiased EDEN (DRIVE).
- While EDEN/DRIVE and TurboQuant share common analytical foundations like random rotations and Lloyd-Max quantization, EDEN's optimized scale parameter selection leads to superior theoretical bounds and empirical performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.