Reproduce “SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controls” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qianxun Xu, Chenxi Song, Yujun Cai, Chi Zhang
SwitchCraft introduces a training-free framework that enables text-to-video diffusion models to generate multi-event videos by dynamically steering attention at the frame level. This approach ensures all events appear in sequence with smooth transitions and consistent visual quality, outperforming existing baselines in human evaluations for transition smoothness and visual quality, and achieving superior CLIP-T scores of 0.275 for text alignment.
Key insights
- Dynamically modifying frame-specific queries within cross-attention, rather than global attention, allows for precise temporal control over event sequences in T2V generation.
- Explicitly enhancing alignment with target event prompts while simultaneously suppressing alignment with competing events is crucial for clear event separation and smooth transitions.
- An adaptive, inference-time strength solver is necessary to balance effective event steering with the preservation of visual fidelity and temporal consistency, preventing both insufficient control and over-steering artifacts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.