Reproduce “Language Driven Occupancy Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhu Yu, Bowen Pang, Lizhe Liu, Runmin Zhang, Qiang Li, Si-Yuan Cao, Maochun Luo, Mingxia Chen and 2 more
The Language Driven Occupancy Prediction (LOcc) framework introduces a method for automatically creating accurate, dense, and fine-grained 3D language occupancy ground truth. This approach allows existing occupancy models to achieve superior zero-shot performance in perceiving 3D scenes with arbitrary vocabularies, significantly reducing reliance on costly manual 3D semantic annotations.
Key insights
- The quality of pseudo-labeled 3D language ground truth directly correlates with the performance of open-vocabulary occupancy models.
- A semantic transitive labeling pipeline, which meticulously transfers 2D language features to 3D space via LiDAR points and majority-voting voxelization, effectively generates robust 3D supervision.
- Existing 3D occupancy network architectures can be effectively adapted for open-vocabulary tasks by replacing their prediction heads and training with accurately generated language-driven pseudo-labels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.