Reproduce “CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Liu, Xiangyang Yin, Tianhua Xia, Haiyu Wang, Sai Qian Zhang
CodeQuant2 quantizes reasoning models while targeting lower-cost inference without losing benchmark accuracy. It calibrates on general text and early generated reasoning tokens, smooths activation outliers, allocates weight clusters across matrices, and uses a lookup-table GPU kernel. Under A4W4, it averages 0.770 across six accuracy benchmarks on Qwen3-4B-Thinking, versus 0.664 for CodeQuant. It reports 2.42× average decoding speedup over BF16 on tested workloads and hardware.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.