Reproduce “SpectrumQuant: Spectrum Alignment for LLM Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula
SpectrumQuant estimates activation covariance from calibration data and derives a closed-form rotation that aligns and then mixes coordinates to spread variance before low-bit quantization. It avoids iterative rotation optimization. Under shared-scale quantization analysis, this can reduce distortion. On Llama-3 8B, its W4-A4-KV4 results are competitive with DartQuant. For the 70B model, calibration takes 43 minutes after activations are pre-saved; this is a calibration result, not an inference speedup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.