Reproduce “ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simon Boeder, Fabian Gigengack, Simon Roesler, Holger Caesar, Benjamin Risse
ShelfOcc creates semantic voxel labels from camera video using pretrained geometric and semantic models, then trains occupancy networks directly on them. It separates static geometry accumulated across frames from dynamic objects placed per frame. On Occ3D-nuScenes validation, ShelfOcc-trained occupancy reaches 22.87 mIoU and 56.14 geometric IoU, versus 17.08 and 46.91 for GaussianFlowOcc; this cross-method comparison does not isolate supervision alone.
Key insights
- Direct 3D supervision from high-quality, FM-generated pseudo-labels significantly enhances the geometric understanding of occupancy networks compared to methods relying on ambiguous 2D signals.
- Robust handling of dynamic objects and temporal aggregation of static elements are critical for generating consistent and accurate 3D pseudo-labels in complex, dynamic driving environments.
- Refining 2D semantic masks with 'sky grounding' and applying confidence filtering to 3D point clouds effectively reduces noise and improves the overall quality of generated pseudo-labels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.