Reproduce “MotionClone: Training-Free Motion Cloning for Controllable Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang and 1 more
MotionClone transfers motion from a reference video into a prompted video by retaining the strongest frame-to-frame temporal-attention links and guiding generation without motion-specific fine-tuning. It extracts this representation with one noise-adding and denoising step, while video generation remains iterative. In two qualitative examples, sparse control transfers motion better than uniform full-attention control; subtle local motion and overlapping objects remain limitations.
Key insights
- Temporal attention maps within video generation models inherently encode rich motion information that can be leveraged directly for motion cloning.
- Focusing on sparse, dominant components of temporal attention maps is crucial for effective motion synthesis and disentangling motion from structure.
- Motion guidance is most effective when applied strategically in early denoising steps and specific temporal attention layers, ensuring strong motion alignment without degrading visual quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.