Reproduce “DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junpeng Jiang, Gangyi Hong, Miao Zhang, Hengtong Hu, Kun Zhan, Rui Shao, Liqiang Nie
The DiVE framework generates high-fidelity, temporally coherent, and cross-view consistent multi-view driving videos, controllable through bird's-eye view layouts and textual descriptions. It achieves state-of-the-art Fréchet Video Distance (FVD) scores and improves downstream perception metrics (mIoU, mAP) while accelerating inference by 2.62x.
Key insights
- A unified cross-attention mechanism can effectively integrate heterogeneous conditioning data, such as text, 3D objects, and camera parameters, for complex scene generation.
- Distilling guidance information into auxiliary branches (Multi-Control Auxiliary Branch Distillation) significantly reduces the computational overhead of Classifier-Free Guidance in multi-conditional diffusion models.
- Progressive resolution scaling during inference (Resolution Progressive Sampling) can accelerate high-resolution video generation by performing initial steps at reduced resolutions without sacrificing output quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.