Reproduce “RespoDiff: Dual-Module Bottleneck Transformation for Responsible & Faithful T2I Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Silpa Vadakkeeveetil Sreelatha, Sauradip Nag, Muhammad Awais, Serge Belongie, Anjan Dutta
RespoDiff introduces a framework for text-to-image generation that effectively reduces societal biases and inappropriate content in outputs while preserving high image quality and semantic alignment with input prompts. This method achieved a gender bias reduction to a DevRat of 0.14, significantly lower than baseline models like Stable Diffusion's 0.68, without compromising visual fidelity.
Key insights
- Intervening in the intermediate bottleneck layer of UNet architectures provides an effective semantic latent space for targeted modifications to T2I outputs.
- Decomposing the transformation into dedicated modules (RAM for responsibility, SAM for semantic preservation) allows for simultaneous optimization, overcoming the traditional trade-off between fairness/safety and generative quality.
- Coordinated score-matching objectives enable the framework to align generated content with specific responsible concepts while regularizing the overall transformation to maintain fidelity to the original model's behavior for neutral prompts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.