Reproduce “DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weijie Wang, Jiagang Zhu, Zeyu Zhang, Xiaofeng Wang, Zheng Zhu, Guosheng Zhao, Chaojun Ni, Haoxiao Wang and 9 more
DriveGen3D, developed by researchers from Zhejiang University, Tsinghua University, Peking University, Chinese Academy of Sciences, Monash University, and GigaAI, presents an efficient framework for generating high-quality dynamic 3D driving scenes from multimodal inputs. The system reduces end-to-end generation time to under 6 minutes, improving upon prior methods, and achieves competitive 3D reconstruction quality even with synthetic inputs.
Key insights
- Accelerating diffusion models by applying caching strategies solely to the conditional branch significantly reduces inference time (up to 3.3x speedup) with minimal impact on perceptual quality.
- Targeted 8-bit quantization in cross-view attention, specifically for the Value (V) matrices due to their smaller numeric range, offers additional computational savings without notable degradation.
- Integrating a temporal window of frames into feed-forward 3D reconstruction allows the model to capture dynamics and maintain high spatial fidelity and temporal coherence in large-scale scenes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.