Reproduce “Comp-Attn: Present-and-Align Attention for Compositional Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hongyu Zhang, Yufan Deng, Shenghai Yuan, Yian Zhao, Peng Jin, Xuehan Hou, Chang Liu, Jie Chen

Comp-Attn introduces a training-free framework to enhance compositional video generation by simultaneously addressing subject presence and inter-subject relational alignment within diffusion models. It demonstrated performance gains of up to 29.84% on compositional T2V benchmarks and 5.36% on T2I benchmarks, while incurring only a 5.1% increase in inference time.

Key insights

  • Decoupling the compositional generation problem into subject presence (addressed at the conditioning level) and relational alignment (addressed at the attention distribution level) allows for more effective and efficient solutions.
  • Adaptive interpolation of subject-specific conditions based on their saliency prevents semantic suppression and subject omission in complex prompts.
  • Dynamically modulating cross-attention using an IoU-guided feedback mechanism, comparing planned layouts with current attention maps, offers a flexible and robust way to enforce spatial relationships without excessive rigidity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.