Reproduce “ClickSeg: 3D Instance Segmentation with Click-Level Weak Annotations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Leyao Liu, Tao Kong, Minzhao Zhu, Jiashuo Fan, Lu Fang

ClickSeg enables 3D instance segmentation using only one arbitrary click per object instance, drastically reducing annotation costs. The method achieves performance levels close to fully-supervised approaches, reaching 90% of their mAP@50 on ScanNetV2 with 0.02% supervision, and sets new benchmarks for weakly supervised 3D instance and semantic segmentation.

Key insights

  • High-quality 3D instance segmentation can be achieved with extremely sparse click-level supervision (as low as 0.02% of full labels) through a carefully designed training process.
  • Using k-means clustering with annotated points as fixed seeds for online pseudo-label generation effectively propagates sparse supervision to unlabeled points, enabling robust learning of instance features and offsets.
  • Combining embedding distance, semantic similarity, and spatial distance in the clustering metric is crucial for generating accurate pseudo-labels and resolving ambiguities in weakly supervised scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.