Reproduce “EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models” on OpenResearch

About this paper

Yefei He, Jing Liu, Weijia Wu, Hong Zhou, Bohan Zhuang

EfficientDM, a data-free and quantization-aware fine-tuning framework, enables the practical deployment of low-bit diffusion models on resource-constrained devices. It achieves state-of-the-art performance for 4-bit weight and activation (W4A4) quantization, reducing model size by 6.58x and accelerating inference by 3.34x compared to full-precision, all without requiring original training data.

Key insights

• Effective low-bit quantization for diffusion models requires adapting parameter-efficient fine-tuning (PEFT) methods to be truly quantization-aware for seamless deployment.
• High-quality low-bit quantization can be achieved without original training data by distilling knowledge from a full-precision model using synthetically generated input noise.
• Addressing the dynamic nature of activation distributions across denoising steps with temporal-aware quantization scales is crucial for maintaining performance at aggressive bit-widths.

Summary by alphaXiv.