Reproduce “ASSIST-3D: Adapted Scene Synthesis for Class-Agnostic 3D Instance Segmentation” on OpenResearch

About this paper

Shengchao Zhou, Jiehong Lin, Jiahui Liu, Shizhen Zhao, Chirui Chang, Xiaojuan Qi

ASSIST-3D, a novel 3D scene synthesis pipeline from The University of Hong Kong, generates diverse, complex, and plausible synthetic scenes to enhance class-agnostic 3D instance segmentation. It improves the generalization capabilities of models like Mask3D, achieving an AP of 22.2 on ScanNet++ and 29.0 on S3DIS, surpassing prior methods by up to 2.6 and 3.3 AP respectively on out-of-domain benchmarks.

Key insights

• Effective synthetic data for class-agnostic 3D instance segmentation must prioritize geometry diversity, context complexity (breaking typical co-occurrence), and layout reasonability to enhance model generalization.
• Leveraging LLMs for high-level spatial reasoning can guide the generation of physically plausible 3D scene layouts, overcoming limitations of purely random or rule-based placement.
• Mimicking real-world sensor acquisition processes during synthetic point cloud generation is crucial for reducing the domain gap and making synthetic data effective for training.

Summary by alphaXiv.