Reproduce “ChildDiffusion: Unlocking the Potential of Generative AI and Controllable Augmentations for Child Facial Data using Stable Diffusion and Large Language Models” on OpenResearch

About this paper

Muhammad Ali Farooq, Wang Yao, Peter Corcoran

Researchers at the University of Galway developed ChildDiffusion, a framework that adapts Stable Diffusion models using existing GAN-generated data, Dreambooth, LoRA, and ControlNet to create photorealistic and controllable synthetic child facial data. The system enables diverse augmentations through text prompts and provides an open-sourced dataset of synthetic child faces for ethical AI research, demonstrating robust quality with competitive CLIP and FID scores.

Key insights

• Latent diffusion models, when appropriately fine-tuned and augmented with control mechanisms, offer superior photorealism, diversity, and controllability compared to GANs for generating sensitive demographic data like child faces.
• Leveraging outputs from prior generative models (e.g., GANs) as seed data for fine-tuning diffusion models provides an effective strategy for domain adaptation and enhancing data realism.
• Combining textual prompts with visual conditioning inputs (via ControlNet) enables precise and complex augmentations, allowing for the generation of highly specific and diverse synthetic datasets that address real-world data scarcity and privacy concerns.

Summary by alphaXiv.