Reproduce “VISTA: Open-Vocabulary, Task-Relevant Robot Exploration with Online Semantic Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Keiko Nagami, Timothy Chen, Javier Yu, Ola Shorinwa, Maximilian Adang, Carlyn Dougherty, Eric Cristofalo, Mac Schwager
VISTA facilitates open-vocabulary, task-relevant robot exploration in unknown environments by building real-time semantic 3D Gaussian Splatting maps. It employs a unified information gain metric that combines geometric diversity and semantic relevance, demonstrating enhanced success rates in locating hidden objects on both quadrotor and quadruped robots.
Key insights
- A unified information gain metric that combines geometric uncertainty with semantic relevance leads to more robust and efficient task-relevant robot exploration than methods relying on mode switching.
- Real-time semantic 3D Gaussian Splatting provides a sufficiently high-fidelity and efficient scene representation for online robotic mapping and open-vocabulary semantic understanding.
- Gradually decaying the weight of geometric gain in an MPC loop effectively balances the robot's focus between general environment exploration and task-driven exploitation for object search.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.