Reproduce “DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haokun Lin, Xinle Jia, Haobo Xu, Bingchen Yao, Xianglong Guo, Yichen Wu, Zhichao Lu, Ying Wei and 2 more
DuQuant++ introduces an effective post-training quantization method for Large Language Models using the MXFP4 format, employing an outlier-aware fine-grained rotation aligned with MXFP4's block structure. The approach consistently achieves state-of-the-art performance on LLaMA-3 models, significantly reducing perplexity and improving zero-shot accuracy with zero online inference overhead.
Key insights
- The independent scaling factors in MXFP4's block-wise design simplify the rotation pipeline, allowing a single fine-grained, outlier-aware rotation to suffice within each block, unlike more complex schemes needed for integer quantization.
- Aligning the rotation block size with the MXFP4 group size (32 elements) is crucial for effectively mitigating outlier-induced quantization error at a fine-grained level.
- Combining fine-grained outlier-aware rotation with second-order weight compensation (like GPTQ) provides complementary benefits, further enhancing overall quantization performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.