Reproduce “Avi: Action from Volumetric Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Harris Song, Long Le
Avi is a 3D Vision-Language Action (VLA) model that shifts robotic action generation to predicting desired 3D geometric changes in manipulated objects, using a "language-to-geometry" approach. This architecture leverages 3D point clouds and classical geometric transformations, achieving strong performance and robustness in complex manipulation tasks compared to action-token-based baselines.
Key insights
- Decoupling high-level geometric goals from low-level robot actuation, by predicting 3D volumetric changes, increases the system's robustness to environmental variations and improves generalizability.
- Shifting from opaque action tokens to explicit 3D geometric outcomes enhances the interpretability of a robot's intended actions.
- A novel location quantization technique effectively adapts 3D Multi-Modal Large Language Models, originally trained on single objects, to reason about object-level spatial context in multi-object robotic scenes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.