Reproduce “SAM-guided Graph Cut for 3D Instance Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoyu Guo, He Zhu, Sida Peng, Yuang Wang, Yujun Shen, Ruizhen Hu, Xiaowei Zhou
A 3D instance segmentation framework utilizes a "3D-to-2D query" mechanism, integrating the generalization capabilities of 2D models like SAM with 3D geometric information through a graph cut approach. This method, from Zhejiang University, Beijing Normal University, Ant Group, and Shenzhen University, demonstrates enhanced performance and generalization across diverse 3D datasets, requiring no manual 3D annotations for training.
Key insights
- Generalization of large 2D vision models like SAM can be effectively transferred to 3D by using a 3D-to-2D query mechanism.
- Framing 3D instance segmentation as a graph cut problem on superpoints provides a robust way to integrate multi-view 2D semantic information and 3D geometry.
- Training a Graph Neural Network with pseudo-labels derived from 2D segmentation networks enables improved 3D segmentation performance and cross-dataset generalization without costly manual 3D annotations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.