Reproduce “SHq: Learnable Grids for LLM Weight Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Michael Helcig, Earl Killian, Dan Alistarh
The authors introduce SHq, a lookup-table quantization method that selects a two-parameter grid for each model and bit width using full-vocabulary next-token-distribution KL against its BF16 counterpart. Its adjustable spacing and asymmetry let a signed block scale choose the grid or its mirror. SHq has the lowest held-out KL in 45 of 50 tested settings, but not universally.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.