Reproduce “ClickSeg: 3D Instance Segmentation with Click-Level Weak Annotations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leyao Liu, Tao Kong, Minzhao Zhu, Jiashuo Fan, Lu Fang
ClickSeg enables 3D instance segmentation using only one arbitrary click per object instance, drastically reducing annotation costs. The method achieves performance levels close to fully-supervised approaches, reaching 90% of their mAP@50 on ScanNetV2 with 0.02% supervision, and sets new benchmarks for weakly supervised 3D instance and semantic segmentation.
Key insights
- High-quality 3D instance segmentation can be achieved with extremely sparse click-level supervision (as low as 0.02% of full labels) through a carefully designed training process.
- Using k-means clustering with annotated points as fixed seeds for online pseudo-label generation effectively propagates sparse supervision to unlabeled points, enabling robust learning of instance features and offsets.
- Combining embedding distance, semantic similarity, and spatial distance in the clustering metric is crucial for generating accurate pseudo-labels and resolving ambiguities in weakly supervised scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.