Reproduce “MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Penghui Liu, Jiangshan Wang, Yutong Shen, Shanhui Mo, Chenyang Qi, Yue Ma
MultiMotion introduces a framework for multi-object video motion transfer via a Video Diffusion Transformer, achieving precise object-specific motion control and improved temporal consistency. The framework integrates a Mask-aware Attention Motion Flow and a high-order RectPC solver. It quantitatively outperforms existing methods, demonstrating improved motion fidelity (0.985) and temporal consistency (0.978), and includes the MultiMotionEval benchmark dataset for evaluation.
Key insights
- Explicitly disentangling motion features for individual objects within Diffusion Transformers through mask-aware attention significantly enhances controllability and mitigates motion entanglement.
- The use of a high-order predictor-corrector solver, reparameterized in a lambda-space, leads to more stable and accurate diffusion inversion, particularly beneficial for generating complex multi-entity videos.
- Dynamically adjusting an object's motion loss contribution based on its visibility and interactions, via an adaptive weighting mechanism, improves the robustness of multi-object guidance in occluded or complex scenes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.