Reproduce “Temporal Dynamic Quantization for Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junhyuk So, Jungwon Lee, Daehyun Ahn, Hyungjun Kim, Eunhyeok Park
This work introduces Temporal Dynamic Quantization (TDQ), a method for efficiently quantizing diffusion models that adapts to the unique temporal variation of their activation distributions. The approach allows high-performance diffusion models to be deployed on resource-limited devices by maintaining high generative quality even at low bit-widths without incurring additional inference overhead.
Key insights
- Diffusion models exhibit significant temporal variation in activation distributions throughout their iterative denoising process, unlike conventional neural networks.
- The known time step in the diffusion process can serve as a reliable input to generate optimal, dynamic quantization intervals, eliminating the need for runtime statistics.
- Geometric Fourier encoding of the time step is crucial for the learning network to effectively capture high-frequency dynamics and ensure stable training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.