Reproduce “InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuoran Yang, Xi Guo, Chenjing Ding, Chiyu Wang, Wei Wu, Yanyong Zhang
InstaDrive, an instance-aware driving world model, generates realistic and consistent driving videos by explicitly addressing instance-level temporal consistency and spatial geometric fidelity. The model achieves state-of-the-art video quality, significantly reduces object ID switches in tracking, and improves perception model performance, including a 3.6-point increase in NDS when used for data augmentation.
Key insights
- Explicitly guiding the diffusion process with instance flow information and unique track IDs is effective for maintaining instance identity and appearance over video sequences.
- Accurate 3D bounding box projection combined with depth-based occlusion reasoning is essential to achieve high spatial geometric fidelity and correct object interaction in generated scenes.
- Procedural generation of diverse control conditions from simulators like CARLA enables the creation of realistic, safety-critical long-tail scenarios, enhancing training data for autonomous systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.