Reproduce “LOSC: LiDAR Open-voc Segmentation Consolidator” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nermin Samet, Gilles Puy, Renaud Marlet
Researchers from Valeo.ai developed LOSC (LiDAR Open-voc Segmentation Consolidator), an annotation-free method for zero-shot open-vocabulary semantic and panoptic segmentation of LiDAR data in driving environments. LOSC leverages 2D Vision-Language Models and refines pseudo-labels through spatio-temporal and augmentation-based consolidation, achieving state-of-the-art performance with 49.3 mIoU on nuScenes and 35.2 mIoU on SemanticKITTI for semantic segmentation.
Key insights
- Combining spatio-temporal consistency through majority voting in voxels and robustness to image augmentations effectively denoises and consolidates noisy 2D VLM-derived 3D pseudo-labels.
- A self-supervised pretrained 3D network provides a strong class-agnostic foundation, significantly boosting performance when finetuned with refined pseudo-labels.
- Iterative finetuning of the 3D model, where it generates new pseudo-labels that are re-consolidated with TBC, consistently improves label quality and expands coverage across the dataset.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.