Reproduce “DIVE: Taming DINO for Subject-Driven Video Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Huang, Wei Xiong, He Zhang, Chaoqi Chen, Jianzhuang Liu, Mingfu Yan, Shifeng Chen
Researchers from SIAT, NVIDIA, and Adobe Research developed DIVE, a framework for subject-driven video editing that leverages DINOv2 features for robust motion and identity guidance. This approach achieves high-quality, temporally consistent video edits with accurate subject replacement, outperforming previous methods and earning strong user preference.
Key insights
- DINOv2 features effectively disentangle subject motion from appearance, providing robust semantic correspondences that prevent source "bleed-through".
- Their semantic sparsity enables accurate motion alignment without the visual artifacts caused by dense auxiliary guidance.
- Combining DINO features with LoRAs offers an efficient and precise mechanism for subject identity registration, overcoming biases from the base diffusion model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.