Reproduce “TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dipkumar Patel
Researchers at LLMs Research Inc. introduce TurboAngle, a KV cache compression method that achieves near-lossless quality for large language models without calibration data by transforming activations into a uniformly angularly distributed space. The approach minimizes perplexity degradation by 7-21 times compared to calibration-based methods at moderately higher bit rates, demonstrating effectiveness across diverse LLM architectures.
Key insights
- After a randomized FWHT, consecutive pairs of transformed KV cache elements approximate a spherically symmetric 2D Gaussian, resulting in a uniform angular distribution that allows for effective, calibration-free uniform angle quantization.
- LLM layers exhibit diverse and non-uniform sensitivities to quantization, with early layers often being more sensitive, and some specific layers even demonstrating "negative transfer" where increased precision degrades performance.
- Key (K) cache norms are significantly more sensitive to quantization error (10-20 times) than Value (V) cache norms, necessitating asymmetric bit allocation for optimal quality-rate trade-offs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.