Reproduce “Leech Lattice Vector Quantization for Efficient LLM Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough, Markus Nagel
A framework for efficient large language model compression, Leech Lattice Vector Quantization (LLVQ) from Qualcomm AI Research, leverages the 24-dimensional Leech lattice to achieve state-of-the-art performance at 2 bits per weight across various LLM architectures. This codebook-free approach also enables robust post-training quantization without requiring Hadamard rotations.
Key insights
- The Leech lattice's optimal sphere packing density and symmetry in 24 dimensions provide theoretically superior rate-distortion performance for vector quantization, translating to better empirical results for LLM compression.
- A codebook-free approach, which avoids explicit storage and search by leveraging the inherent structure and hierarchy of lattice shells, makes high-dimensional VQ practically feasible for LLMs.
- Reinterpreting PTQ scale corrections as a retraction onto constant-norm manifolds (Spherical GPTQ) allows for robust quantization without computationally intensive Hadamard rotations, simplifying the deployment pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.