Reproduce “VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohan Liu, Zhihao Gu, Xuanyu Chen, Haitian Zhang, Kaimin Mao, Yan Wu, Wei-Yun Yau, Lin Wang
VistaVLA gives a vision-language-action policy a compact 3D scene representation that links object meaning with spatial layout. It attaches language-aligned features to 3D Gaussian primitives, compressing roughly 100,000 primitives into 64 policy tokens. Across seven tabletop tasks, it averaged 75.7% success versus 52.9% for VLA-Adapter, under calibrated multi-view evaluation on one fixed robot platform with calibrated multi-view cameras.
Key insights
- Integrating a geometry- and semantics-aware 3D representation, built from 3D Gaussian primitives, provides VLA policies with a more robust and spatially aware understanding of the scene.
- Efficiently compressing high-dimensional 3D-grounded semantic information into a compact set of tokens is crucial for real-time robotic control while preserving action-relevant context.
- A multi-stage training approach combining RGB-D reconstruction with teacher-feature distillation effectively imbues 3D Gaussian primitives with both accurate geometry and high-level semantics from foundation models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.