Reproduce “FAST3DIS: Feed-forward Anchored Scene Transformer for 3D Instance Segmentation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Changyang Li, Xueqing Huang, Shin-Fang Chng, Huangying Zhan, Qingan Yan, Yi Xu

FAST3DIS introduces an end-to-end, feed-forward framework for multi-view 3D instance segmentation that directly predicts 3D-consistent binary masks from unposed images. The method achieves over 115x to 250x inference speedup compared to clustering-based approaches while improving segmentation quality for small and adjacent objects.

Key insights

  • Directly predicting 3D instance masks using learned 3D anchors and an anchor-sampling attention mechanism fundamentally bypasses the scalability bottlenecks of post-hoc clustering, leading to orders-of-magnitude inference speedups.
  • Explicit regularization, combining view-consistent contrastive loss and a dynamically scheduled spatial overlap penalty, is critical for learning distinct instance boundaries and resolving query collisions in class-agnostic 3D segmentation.
  • Integrating 3D geometry information directly into the feature extraction pipeline and pixel decoder significantly enhances 3D awareness and the quality of instance mask generation from multi-view inputs without intermediate depth supervision.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.