Reproduce “Quantization of Large Models with Weight Outliers and Sparsity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonidas Tsigkounakis, Abraham Jaeger Mountain, Yilang Zhang, Georgios B. Giannakis
The authors introduce a zero-shot post-training quantization method that combines a low-bit weight backbone with high-precision corrections for selected error-heavy rows and broader low-rank error patterns, without calibration examples or fine-tuning data. Using simulated annealing, the method improves Llama 3.1 8B’s NF4 GSM8K exact-match accuracy to 46.32%, versus 44.96% without row sparsity; gains vary by task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.