Reproduce “AID: Attention Interpolation of Text-to-Image Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiyuan He, Jinghao Wang, Ziwei Liu, Angela Yao
A training-free framework called Attention Interpolation via Diffusion (AID) enhances conditional image interpolation in text-to-image diffusion models by directly manipulating attention mechanisms and employing non-uniform interpolation coefficients. This approach yields sequences with improved thematic consistency (e.g., up to 23.9% better), smoother visual transitions, and higher fidelity (e.g., 66.62 FID reduction) compared to prior text embedding interpolation methods.
Key insights
- Attention mechanisms, especially self-attention, are critical for coherent spatial layout during interpolation, and simply interpolating text embeddings provides insufficient control.
- Fusing interpolated attention features with directly generated features from interpolated latent seeds is essential for maintaining image fidelity and preventing artifacts.
- Perceptually smooth transitions in image interpolation are better achieved through non-uniform sampling of interpolation coefficients, which can be managed by distributions like the Beta distribution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.