Reproduce “Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiang Li, Pengfei Li, Yupeng Zheng, Wei Sun, Yan Wang, Yilun Chen
A semi-supervised vision-centric 3D occupancy world model, PreWorld, learns future scene dynamics for autonomous driving by effectively using 2D labels for temporal self-supervision. It sets new benchmarks on the Occ3D-nuScenes dataset, achieving 34.69% mIoU for 3D occupancy and 9.55% mIoU for 4D occupancy forecasting, while improving self-supervised 4D forecasting mIoU by 1300%.
Key insights
- Direct forecasting of volume features improves information preservation and enables end-to-end optimization in 4D occupancy world models.
- Temporal 2D rendering provides effective self-supervision from readily available 2D labels for learning 3D geometry, semantics, and future scene dynamics.
- A two-stage training approach combining self-supervised pre-training and fully-supervised fine-tuning boosts data efficiency and scalability for complex perception tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.