Reproduce “GAUDA: Generative Adaptive Uncertainty-guided Diffusion-based Augmentation for Surgical Segmentation” on OpenResearch

About this paper

Yannik Frisch, Christina Bornberg, Moritz Fuchs, Anirban Mukhopadhyay

GAUDA is introduced as a method for surgical semantic segmentation, employing a latent diffusion model to generate high-quality image-mask pairs that adaptively augment training based on a downstream model's epistemic uncertainty. This approach improved segmentation performance by an average absolute IoU of 1.5-1.6% across two surgical datasets, especially benefiting less frequent classes.

Key insights

• Compressing high-dimensional image and mask data into a compact latent space with VQ-GANs enables computationally efficient and semantically coherent joint (image, mask) synthesis for diffusion models.
• Guiding generative augmentation by a segmentation model's epistemic uncertainty effectively targets specific learning deficiencies, yielding greater improvements, especially for classes initially performing poorly.
• Integrating data generation adaptively during the training loop, rather than fixed pre-training, ensures that synthetic samples are optimally aligned with the model's evolving learning needs, promoting faster convergence and better generalization.

Summary by alphaXiv.