Reproduce “FraQAT: Quantization Aware Training with Fractional bits” on OpenResearch

About this paper

Luca Morreale, Alberto Gil C. P. Ramos, Malcolm Chadwick, Mehid Noroozi, Ruchika Chavhan, Abhinav Mehrotra, Sourav Bhattacharya

FraQAT introduces a Quantization-Aware Training (QAT) method that leverages fractional bit-widths and curriculum learning to enable large generative models, such as Diffusion Transformers, to operate at aggressive W4A8 precision. This approach achieves high-fidelity image generation and efficient deployment on edge devices like smartphones.

Key insights

• Gradually reducing precision with fractional bits creates a smoother optimization landscape, allowing weights to adjust to stable quantized configurations and more effectively mitigating outlier issues than direct or integer-only quantization.
• The curriculum learning approach, which starts with coarser quantization and incrementally moves to finer, lower-bit steps, is crucial for maintaining generative model quality during aggressive quantization.
• FraQAT's methodology demonstrates broad applicability, proving effective on various large Text-to-Image diffusion models and also showing minimal quality drop when applied to Large Language Models.

Summary by alphaXiv.