Reproduce “IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhongping Ji
IsoQuant introduces a blockwise feature decorrelation method for large language model KV cache compression, employing hardware-aligned SO(4) isoclinic rotations in 4D blocks. This approach achieved average kernel-level speedups exceeding 4x compared to previous methods while maintaining reconstruction quality.
Key insights
- Leveraging the Lie-theoretic decomposition of SO(4) allows for a mathematically principled and compact representation of 4D rotations, offering high degrees of freedom for local mixing.
- Aligning the block size (4D) with typical power-of-two feature dimensions in transformers dramatically improves hardware efficiency by optimizing memory access patterns and enabling straightforward SIMD processing.
- Integrating the entire per-block transform (rotation, quantization, inverse rotation) into fused CUDA kernels minimizes off-chip memory access and keeps data in registers, leading to substantial real-world latency reductions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.