Reproduce “Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction” on OpenResearch

About this paper

Jatin Chhugani, Geonhwa Jeong, Bor-Yiing Su, Yunjie Pan, Hanmei Yang, Aayush Ankit, Jiecao Yu, Summer Deng and 3 more

The authors propose software scaling adjustments, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS), to improve MXFP4 quantization without changing its four-bit values or requiring hardware modifications. They evaluate direct-cast quantization without calibration or fine-tuning. On Llama 3.1-8B-Instruct, OAS+MBS scores 66.50% averaged across six tasks, 0.52 points below NVFP4; this is not parity on every task or model.

Key insights

• Reducing MXFP4 block size from 32 to 16 is hardware-inexpensive (2% tensor core area overhead) and recovers approximately 1 dB in Quantization Signal-to-Noise Ratio (QSNR).
• Increasing scaling factor mantissa precision (e.g., from E8M0 to E4M3) offers a 3-4 dB QSNR improvement but incurs a high hardware cost (12.6% total tensor core area overhead).
• Software-only techniques like OAS and MBS can effectively address MXFP4's fidelity limitations (coarse block granularity and power-of-two scaling precision) on existing hardware without architectural modifications.

Summary by alphaXiv.