Reproduce “On Data Augmentation for GAN Training” on OpenResearch

About this paper

Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Trung-Kien Nguyen, Ngai-Man Cheung

Researchers introduced Data Augmentation Optimized for GAN (DAG), a framework designed to integrate data augmentation into GAN training while theoretically preserving the original data distribution, leading to improved sample quality and training stability, particularly with limited datasets. The method achieved an FID score of 23.7 on CIFAR-10 using Dist-GAN with FlipRot augmentation, compared to a baseline of 29.6, and substantially enhanced performance on medical images with scarce data.

Key insights

• The Jensen-Shannon divergence between two distributions remains invariant under differentiable and invertible transformations, providing a theoretical basis for applying certain data augmentations without altering the generator's target distribution.
• Training multiple discriminators, each on a different (potentially augmented) view of the data, and having the generator learn to fool all of them simultaneously, provides richer supervisory signals and helps preserve the original data distribution.
• Regularizing discriminators through shared weights across their initial layers, while keeping final classification heads separate for each augmented view, significantly enhances performance and feature learning without compromising the JS-preserving property.

Summary by alphaXiv.