Reproduce “LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection” on OpenResearch

About this paper

Liulu He, XuanAng Liu, Juntao Liu, Taolue Feng, Ting Lu, Chunsheng Gan, Zhiyv Peng, Yuan Du and 3 more

LiftQuant introduces a framework for continuous bit-width control in Large Language Model quantization, employing a dimensional lifting and projection mechanism to generate non-uniform codebooks from 1-bit lattices. This approach enables Pareto-optimal deployment by precisely matching model memory footprints to diverse hardware VRAM capacities, demonstrating superior performance at fractional bit-widths compared to integer-constrained baselines.

Key insights

• Linear projections of a high-dimensional 1-bit lattice can generate dense, non-uniform codebooks in lower dimensions, mimicking the expressive power of Vector Quantization.
• Decoupling the effective bit-width from the underlying coding format allows for continuous precision control while maintaining a hardware-friendly decoding path using only linear transformations and 1-bit uniform quantizers.
• Combining high-dimensional projection with layer-wise whitening and differentiable fine-tuning creates a robust, unified architecture capable of accurately quantizing LLMs to arbitrary fractional bit-widths.

Summary by alphaXiv.