Reproduce “Data Pruning in Generative Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rania Briq, Jiangtao Wang, Stefan Kesselheim

Research from the Jülich Supercomputing Centre demonstrates that data pruning can enhance the efficiency and performance of generative Diffusion Models, challenging the intuition that more data always improves generative tasks. The study shows that strategic data reduction, particularly using clustering methods on large datasets, can lead to improved FID scores and mitigate dataset biases while significantly reducing computational overhead.

Key insights

  • Data pruning can be highly beneficial for training generative Diffusion Models, with strategies showing improved performance and efficiency, particularly on larger datasets.
  • The effectiveness of data pruning methods is dataset-dependent: random pruning performs well on homogeneous data, while clustering-based methods excel on heterogeneous datasets by selecting representative samples.
  • Diffusion Models exhibit remarkable resilience to data reduction compared to other generative architectures (VAEs, GANs), maintaining quality even with significantly pruned datasets due to their iterative refinement and latent space operation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.