Reproduce “LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Li, Minghan Qin, Zhengyu Zou, Diqi He, Xinhao Ji, Bohan Li, Bingquan Dai, Dingewn Zhang and 1 more
LangSurf introduces a Language-Embedded Surface Field represented by Gaussians, enhancing 3D scene understanding by accurately aligning language features with object surfaces and integrating hierarchical contextual information. The framework achieves notable improvements in open-vocabulary 2D and 3D semantic segmentation, surpassing previous methods by up to 10% in mAcc and 8% in mIoU on the LERF dataset.
Key insights
- Enforcing explicit geometric constraints and promoting Gaussian flattening encourages language features to adhere closely to object surfaces, mitigating 'outlier languages'.
- Integrating global image context with multi-scale mask-pooled features provides a richer, more robust semantic representation that addresses low-texture areas and complex object structures.
- A spatial-aware semantic supervision loss, coupled with semantic grouping, improves the consistency of semantic information among proximal Gaussians and clarifies object boundaries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.