Reproduce “Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lechun You, Zhonghua Wu, Weide Liu, Xulei Yang, Jun Cheng, Wei Zhou, Bharadwaj Veeravalli, Guosheng Lin
Integrating SAM Supervision augments sparse 3D annotations with Semantic-SAM image masks projected into point clouds, allowing a 3D-only segmentation model to learn from substantially denser but noisy supervision. With only 20 labeled points per ScanNetV2 scene, it reaches 69.4% mIoU, and under the One-Thing-One-Click setting achieves 64.4% on S3DIS Area 5, improving Point Transformer V3 baselines by 9.3 and 9.7 percentage points, respectively.
Key insights
- The method separates region discovery from class assignment: Semantic-SAM supplies spatial masks, while sparse 3D annotations and the segmentation model determine semantic categories.
- Quality matters more than raw pseudo-label quantity: the proposed masks expand ScanNetV2 supervision to about 5,565.5 labels per scene at 93.0% accuracy, whereas SAM3D produces more labels but only 81.8% accuracy.
- Using RGB only for training preserves 3D-only inference, but propagation remains vulnerable to mask merging, object boundaries, occlusions, and incorrect sparse labels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.