Reproduce “InvDiff: Invariant Guidance for Bias Mitigation in Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Min Hou, Yueying Wu, Chang Xu, Yu-Hao Huang, Chenxi Bai, Le Wu, Jiang Bian
InvDiff is a framework that mitigates unknown biases in pre-trained text-to-image diffusion models by employing invariant learning without requiring explicit bias annotations or unbiased data. The approach achieves superior bias reduction and maintains image generation quality across multiple datasets, also demonstrating applicability to other generative tasks.
Key insights
- Bias mitigation in generative models can be achieved through invariant learning principles, even without explicit environment labels, by leveraging the differential performance of the biased model across inferred data groups.
- Employing a lightweight, dynamically learned guidance module allows for effective bias reduction in pre-trained diffusion models without the computational burden of full model fine-tuning.
- The approach theoretically links bias mitigation to improved out-of-distribution generalization by reducing the upper bound of the generalization error, enhancing model robustness against spurious correlations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.