Reproduce “OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianyu Huang, Runnan Chen, Dongting Hu, Fengming Huang, Mingming Gong, Tongliang Liu
Researchers from the University of Sydney and the University of Melbourne introduced OpenInsGaussian, a framework for open-vocabulary 3D instance Gaussian segmentation that enhances semantic feature integration into 3D Gaussian Splatting. The method achieves this by employing context-aware feature extraction and attention-driven multi-view aggregation, leading to superior performance on ScanNetV2 and LeRF-OVS datasets.
Key insights
- Leveraging the full image context via intermediate VLM feature maps, rather than cropped regions, substantially improves the quality and semantic fidelity of 2D object features.
- An adaptive, attention-based weighting of multi-view features can effectively filter out noise and inconsistencies, leading to a more robust and coherent 3D semantic representation.
- The combination of context-aware feature extraction and intelligent multi-view aggregation is crucial for bridging the gap between rich 2D semantic cues and consistent 3D scene understanding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.