Reproduce “Zero-Shot Visual Grounding in 3D Gaussians via View Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liwei Liao, Xufeng Li, Xiaoyun Zheng, Boning Liu, Feng Gao, Ronggang Wang
A new framework, Grounding via View Retrieval (GVR), introduces zero-shot visual grounding for 3D Gaussian Splatting (3DGS) scenes by reframing the task as 2D retrieval, leveraging pre-trained 2D visual foundation models. The method achieves state-of-the-art performance on LERF-Mask (87.5% Acc, 56.2% IoU) and 3D-OVS (95.4% Acc), while dramatically reducing scene preparation time to seconds and query latency to milliseconds.
Key insights
- Effective 3D visual grounding in 3DGS can be achieved without 3D annotations or per-scene training by transforming the problem into a 2D retrieval task.
- Leveraging readily available and generalizable 2D visual foundation models (SAM, CLIP, Grounding DINO) significantly improves efficiency and generalization.
- A multi-stage approach combining 2D retrieval, 2D-to-3D projection, and multi-view refinement enables accurate and precise segmentation of target Gaussians.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.