Reproduce “Efficient Coarse-to-Fine Diffusion Models with Time Step Sequence Redistribution” on OpenResearch

About this paper

Yu-Shan Tai, An-Yeu

Researchers at National Taiwan University developed Coarse-to-Fine (C2F) denoising and Time Step Sequence Redistribution (TRD) to optimize Diffusion Models. This combined approach achieved near-lossless image generation quality with 80-90% fewer Multiply-Accumulate Operations (MACs) on CIFAR10 and LSUN-Church datasets, making DMs more computationally efficient.

Key insights

• Processing inputs at lower resolutions during the early, highly noisy stages of denoising, where only coarse features are relevant, can significantly reduce computational load without compromising final image quality.
• The optimal transition point from low-resolution to high-resolution denoising can be efficiently identified by monitoring the minimum rank in Principal Component Analysis (PCA) of predicted images, indicating the shift from noise removal to detail generation.
• Effective optimization of time step sequences for Diffusion Models can be achieved rapidly through iterative local search using L2 loss on a small calibration dataset, bypassing the computational expense of extensive global search methods.

Summary by alphaXiv.