Reproduce “GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yichen Liu, Puzhen Yuan, Xiang Zhu, Yanjiang Guo, Jianyu Chen
GALA augments image-based latent actions with 3D end-effector geometry so vision-language-action models learn scene motion and fine-grained hand articulation while retaining embodiment-specific controls. It encodes visual and geometric transitions, using UEMR to make geometric motion codes shareable across hand morphologies. On 24 RoboCasa GR-1 tasks, it reaches 55.7% average success; multi-embodiment co-training reaches 68.3%, but this comparison does not isolate UEMR.
Key insights
- Explicit 3D geometry substantially improves articulation modeling: GALA reduces finger-motion prediction error to 7.585 degrees, compared with 15.113 degrees for the no-point-cloud variant.
- Conditioning geometric reconstruction on the initial end-effector geometry encourages the latent code to describe motion rather than static hand shape, while shared decoding and augmentation improve cross-embodiment retrieval.
- Geometry-aware latent supervision complements visual dynamics: adding heterogeneous embodiment data improves GR-1 RoboCasa performance from 55.7% to 68.3%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.