Reproduce “3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihao Xiao, Longlong Jing, Shangxuan Wu, Alex Zihao Zhu, Jingwei Ji, Chiyu Max Jiang, Wei-Chih Hung, Thomas Funkhouser and 4 more
The method enables 3D open-vocabulary panoptic segmentation by fusing learnable LiDAR features with frozen camera CLIP features and training class embeddings comparable with text. In nuScenes B12/N4 validation, with training labels limited to base classes, vegetation’s panoptic quality rises from 0.5 for PFC to 35.2. The result applies to benchmark splits with limited category annotations, not arbitrary recognition.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.