Reproduce “GeoProp: Grounding Robot State in Vision for Generalist Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guoyang Zhao, Quanhao Qian, Gongjie Zhang, Wenhao Li, Jiuniu Wang, Xiaowei Lu, Deli Zhao, Ran Xu
GeoProp, a lightweight adapter, addresses the structural decoupling between visual observations and proprioceptive feedback in robot learning by explicitly grounding 3D robot kinematics within 2D visual feature maps. This method consistently improved manipulation policy success rates by up to 8.7 percentage points in simulation and 11.2 percentage points in real-world household tasks, particularly for precision-oriented actions.
Key insights
- Explicit geometric grounding of 3D robot kinematics in 2D visual features serves as an effective inductive bias, enhancing the performance and robustness of robot manipulation policies.
- Ungrounded proprioceptive input can be counterproductive, leading to suboptimal performance, sometimes worse than vision-only baselines, underscoring the need for direct state-vision correspondence.
- The GeoProp adapter is lightweight and plug-and-play, demonstrating compatibility and consistent improvements across diverse policy architectures (Diffusion Policy, π0) and vision backbones with minimal parameter overhead (2-3%).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.