Reproduce “Genie: Show Me the Data for Quantization” on OpenResearch

About this paper

Yongkweon Jeon, Chungman Lee, Ho-young Kim

The GENIE framework introduces a novel post-training zero-shot quantization method that synthesizes data and jointly optimizes quantization parameters, achieving high-quality quantized models within hours. It surpasses existing zero-shot techniques in accuracy, for instance, reaching 53.38% for MobileNetV2 W2A4, and significantly reduces quantization time, completing ResNet-18 ZSQ in 2.73 hours.

Key insights

• Post-Training Quantization (PTQ) algorithms are more effective and efficient for Zero-Shot Quantization with synthetic data than Quantization-Aware Training (QAT)-based methods, especially at lower bit-widths.
• A hybrid data synthesis strategy, combining generator-based approaches with latent vector optimization and 'swing convolution,' creates more informative synthetic data, significantly boosting ZSQ performance.
• Jointly optimizing quantization step size and softbits within a PTQ framework, by treating the base integer matrix as a constant, consistently enhances quantization accuracy across various models and bit-widths.

Summary by alphaXiv.