Reproduce “GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ye Zhu, Kaleb S. Newman, Johannes F. Lutzeyer, Adriana Romero-Soriano, Michal Drozdzal, Olga Russakovsky
Researchers from Laboratoire d’Informatique (LIX), CNRS, École Polytechnique, IPP, Princeton University, and FAIR at Meta introduce Geometry-Aware Spherical Sampling (GASS), a novel method to enhance diversity in text-to-image generation by disentangling prompt-dependent and prompt-independent variations within the CLIP embedding space. GASS consistently improves diversity metrics like Vendi Score and Spherical Spread Score by up to 10% on benchmarks like ImageNet and DrawBench across various T2I models, while preserving image fidelity and semantic alignment.
Key insights
- T2I image diversity can be effectively conceptualized and controlled by decomposing image embeddings into prompt-dependent and prompt-independent orthogonal components within the CLIP embedding space.
- Guiding the T2I generation process by expanding projections along these disentangled axes enables fine-grained control over diverse aspects, such as semantic attributes and background elements, without modifying the input prompt.
- Re-normalizing perturbed embeddings back onto the unit hypersphere after projection expansion is crucial for maintaining the quality and semantic consistency of generated images during diversity enhancement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.