Reproduce “AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shipeng Cao, Biao Qian, Haipeng Liu, Yang Wang, Meng Wang
AI-T2I, a training-free method, enhances text-to-image synthesis by precisely managing cross-attention maps in diffusion models, addressing issues of fragmented object generation and subject mixing. It achieves improved object coherence and distinctness, leading to higher compositional fidelity and better textual alignment across diverse diffusion architectures and downstream tasks.
Key insights
- Both intra-token scattering and inter-token overlap stem from incoherent spatial distributions of attention activations, requiring a unified solution.
- Euclidean distance is more effective than cosine similarity for spatially manipulating attention activations, promoting both aggregation within subjects and isolation between subjects.
- Direct gradient updates to the latent code based on attention map-derived losses during the critical early denoising stages can robustly guide the generative process without additional training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.