Reproduce “Control-DINO: Feature Space Conditioning for Controllable Image-to-Video Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Edoardo A. Dominici, Thomas Deixelberger, Konstantinos Vardis, Markus Steinberger
Control-DINO introduces a method for controllable image-to-video diffusion by conditioning models on dense DINOv3 features through a ControlNet architecture. The approach uses an appearance-decoupled training strategy to enable diverse video transfer and 3D-guided generation tasks, demonstrating enhanced temporal consistency and structural preservation.
Key insights
- Dense self-supervised features (DINOv3) provide a robust, semantically rich, and general-purpose conditioning signal for video diffusion, offering superior structural and temporal consistency.
- Decoupling appearance from conditioning features during training is critical for achieving versatile generative control, allowing for independent manipulation of appearance (e.g., stylization, relighting) while maintaining scene structure.
- DINO features can effectively bridge 3D representations to video diffusion, enabling high-quality video generation from coarse or uncolored 3D inputs by combining geometric priors with semantic features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.