Reproduce “Discovering Divergent Representations between Text-to-Image Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lisa Dunlap, Joseph E. Gonzalez, Trevor Darrell, Fabian Caba Heilbron, Josef Sivic, Bryan Russell
Researchers introduced COMPCON, an evolutionary search algorithm that automatically identifies and describes subtle, input-dependent differences in how text-to-image generative models interpret semantic concepts. COMPCON achieved a 0.60 Top 1 Attribute Score and 0.64 Top 1 Description Score on the ID² benchmark and revealed distinct qualitative divergences and biases in models such as PixArt Alpha and SDXL-Lightning.
Key insights
- Text-to-image models develop distinct "personalities" and implicit semantic associations, leading to varied visual interpretations for similar prompts.
- Training data composition and diffusion backbone architecture play a crucial role in shaping a model's internal representations and leading to specific stylistic and conceptual divergences.
- Automated, data-driven approaches can effectively detect nuanced, input-dependent biases and stereotypes in generative models that are difficult to identify manually or with predefined lists.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.