Reproduce “Ilov3Splat: Instance-Level Open-Vocabulary 3D Scene Understanding in Gaussian Splatting” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Binh Long Nguyen, Kien Nguyen, Sridha Sridharan, Clinton Fookes, Peyman Moghadam

Ilov3Splat introduces a unified framework for instance-level open-vocabulary 3D scene understanding, built on 3D Gaussian Splatting, that augments Gaussians with view-consistent feature fields. The method achieved 55.15% mean accuracy in 3D object selection and 59.29% mean accuracy in 3D instance segmentation, outperforming prior 3D-GS approaches.

Key insights

  • Directly embedding view-consistent language and instance feature fields into 3D Gaussians improves cross-view consistency and spatial precision over 2D projection-based methods.
  • Explicit instance-level features, trained with contrastive learning, enable precise distinction of individual objects, overcoming limitations of point-level semantic associations.
  • Jointly optimizing geometry and semantic representations within a single framework fosters better alignment and consistency between appearance and language features.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.