Reproduce “Style Aligned Image Generation via Shared Attention” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Amir Hertz, Andrey Voynov, Shlomi Fruchter, Daniel Cohen-Or
Researchers at Google Research and Tel Aviv University introduce StyleAligned, a zero-shot, optimization-free method that enables Text-to-Image diffusion models to generate multiple images with consistent stylistic attributes by modulating self-attention layers with Adaptive Instance Normalization. The approach effectively disentangles style from content, producing visually cohesive image sets without requiring fine-tuning or content leakage.
Key insights
- Direct sharing of attention keys and values across images in a batch leads to content leakage and reduced diversity, highlighting the need for a modulated approach.
- Adaptive Instance Normalization (AdaIN) is crucial for effectively transferring style-related feature statistics between attention components, allowing style alignment without content corruption.
- Manipulating self-attention layers at inference time provides a lightweight and powerful control mechanism for style consistency, outperforming optimization-heavy methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.