Reproduce “Test-Time 3D Occupancy Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fengyi Zhang, Xiangyu Sun, Huitong Yang, Zheng Zhang, Zi Huang, Yadan Luo
TT-Occ builds occupancy maps at test time by combining pretrained geometry and semantic models in a time-aware set of 3D Gaussians, then voxelizing them at a chosen resolution. It requires no occupancy-network pretraining or fine-tuning, but runs foundation models during inference. On Occ3D-nuScenes, with OpenSeeD shared, TT-OccCamera reports mIoU 13.4 versus SelfOcc’s 10.5 and RayIoU@4 15.7 versus 12.0 on benchmark labels.
Key insights
- High-quality 3D occupancy prediction can be achieved without task-specific network training by leveraging the strong generalization capabilities of pre-trained Vision Foundation Models.
- A modular, training-free approach enables flexible adaptation to arbitrary voxel resolutions and open-vocabulary semantics at test time, addressing limitations of fixed-vocabulary and resolution-specific models.
- Maintaining temporal consistency through modality-specific Gaussian tracking and accumulation effectively mitigates artifacts from dynamic objects and noisy VFM predictions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.