Reproduce “RL Bootstrapping of OpenVLA-OFT for a Novel Robot Embodiment” on OpenResearch

About this paper

Damir Nurtdinov, Alexei Kornaev, Alexander Maloletov

This research explores aligning a pretrained VLA policy, OpenVLA-OFT, to a novel cable-driven robot embodiment using only reinforcement learning, without requiring embodiment-specific demonstrations. A two-stage RL approach with dense geometric rewards established initial language-conditioned control in simulation, achieving an average 53.50% success rate for directional commands and demonstrating target-approaching behavior for object-conditioned tasks.

Key insights

• RL-only bootstrapping is a viable method for establishing initial language-conditioned control for a truly novel robot embodiment, circumventing the need for costly embodiment-specific demonstration datasets.
• Dense, progress-based geometric rewards are critical for facilitating learning and embodiment alignment in zero-demo settings, providing effective learning signals before consistent task success can be achieved.
• The findings support a staged methodology for VLA adaptation, where initial RL alignment can precede assisted data collection or further imitation learning, potentially reducing overall development costs and time for novel robots.

Summary by alphaXiv.