Reproduce “Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models” on OpenResearch

About this paper

Miaoxi Zhu, Qihuang Zhong, Li Shen, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

A Zero-Shot Sharpness-Aware Quantization (ZSAQ) framework facilitates low-bit quantization for pre-trained language models (PLMs) without requiring original training data. It employs adversarial learning with a feature adaptation module and Sharpness-Aware Minimization (SAM), yielding a +6.98 average score gain over baseline PTQ on BERT-base in a W2A8 setting and improved generalization across NLP tasks.

Key insights

• The feature adaptation module effectively bypasses the technical difficulty of backpropagating gradients through discrete textual tokens, making adversarial data generation feasible for NLP quantization.
• Sharpness-Aware Minimization (SAM) actively guides the quantization process to find flatter, more robust optima, significantly improving the generalization ability of low-precision models on real-world data.
• The SAM-SGA (Sharpness-Aware Minimization - Stochastic Gradient Ascent) alternating optimization algorithm provides a theoretically convergent approach for solving the combined minimax and sharpness-aware objective function.

Summary by alphaXiv.