Reproduce “UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinqiang Yu, Zekun qi, Jiawei He, Wenyao Zhang, Xuchuan Chen, Guaocai Yao, Li Yi, Zhaoxiang Zhang and 1 more
UniPart frames 3D part segmentation as a language-queryable interface for embodied interaction: given a point cloud and a phrase such as “the handle” or “the gripping area,” it predicts the corresponding points without a fixed part taxonomy. Its LangPart-1M dataset and feed-forward point-based architecture improve zero-shot performance over prior methods, while a real-robot study reports 90.5% manually assessed part-segmentation accuracy and 85.0% grasp success on the intended parts across 20 objects.
Key insights
- Both the training data and model design matter: a controlled study reports 47.25 mIoU when UniPart is trained with LangPart-1M, compared with 35.27 from the architecture change alone and 38.42 from the dataset change alone.
- Simple additive language conditioning outperforms the tested cross-attention, multiplication, and concatenation alternatives, reaching 33.56 mIoU versus 29.22, 28.13, and 30.31, respectively, in the ablation.
- Image-to-point cross-modal pretraining improves segmentation on both partial and complete point clouds, suggesting that multi-view semantic alignment transfers to 3D geometry.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.