Reproduce “WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiachen Lu, Ze Huang, Zeyu Yang, Jiahui Zhang, Li Zhang
Researchers at Fudan University developed WoVoGen, a diffusion-based framework that leverages an explicit 4D world volume to synthesize high-quality, consistent multi-camera driving scene videos. This method achieved an FID score of 27.6 and an FVD score of 417.7 on the nuScenes dataset, outperforming previous generative models in terms of realism and temporal coherence.
Key insights
- Explicitly modeling the driving environment as a dense 4D world volume serves as a powerful foundation for achieving strong geometric and semantic consistency in generated scenes.
- A 'panoptic diffusion' approach, where features from multiple camera views are processed jointly, is effective in enforcing inter-sensor coherence across surround-view images.
- Integrating temporal attention layers and conducting dedicated temporal finetuning on these layers significantly improves temporal consistency in generated video sequences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.