Reproduce “Successfully Applying Lottery Ticket Hypothesis to Diffusion Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chao Jiang, Bo Hui, Bohan Liu, Da Yan

Researchers at Visa, University of Tulsa, Carnegie Mellon University, and Indiana University Bloomington successfully applied the Lottery Ticket Hypothesis to Denoising Diffusion Probabilistic Models (DDPMs), identifying sparse subnetworks capable of generating high-quality images. The approach, which includes a novel layer-wise varied sparsity pruning strategy, reduced model FLOPs by 90% while maintaining or improving performance on image generation benchmarks.

Key insights

  • Denoising Diffusion Probabilistic Models (DDPMs) contain "winning tickets"—sparse subnetworks that can match or exceed the performance of their dense counterparts when re-initialized to their original weights.
  • Layer-wise analysis using Centered Kernel Alignment (CKA) revealed that upstream U-Net modules exhibit higher functional similarity and are less tolerant of pruning compared to downstream modules.
  • Adapting pruning ratios based on layer position, with lower sparsity upstream and higher sparsity downstream, is more effective than uniform pruning for diffusion models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.