Reproduce “Effective Data Augmentation With Diffusion Models” on OpenResearch

About this paper

Brandon Trabucco, Kyle Doherty, Max Gurinas, Ruslan Salakhutdinov

This research introduces DA-Fusion, a data augmentation strategy that leverages pre-trained text-to-image diffusion models to generate semantically diverse training data. The method consistently improved few-shot image classification accuracy by up to 15 percentage points, particularly excelling on fine-grain and novel visual concepts like specific weed species, and its effectiveness was validated even when rigorously preventing data leakage from the generative model's training set.

Key insights

• Semantic data augmentation, enabled by adapting pre-trained diffusion models, consistently outperforms traditional augmentation methods, especially in few-shot learning settings.
• The use of Textual Inversion allows off-the-shelf diffusion models to generate diverse variations for fine-grain or entirely novel visual concepts, addressing a limitation of previous generative augmentation approaches.
• Rigorously preventing data leakage from the generative model's vast training set is crucial for validating the true performance gains of generative data augmentation, demonstrating that improvements are from effective augmentation, not pre-existing knowledge.

Summary by alphaXiv.