Reproduce “Bridging Embodiment Gaps: Deploying Vision-Language-Action Models on Soft Robots” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haochen Su, Cristian Meo, Francesco Stella, Andrea Peirone, Kai Junge, Josie Hughes
Researchers from EPFL successfully adapted Vision-Language-Action models, specifically OpenVLA-OFT, to a custom soft continuum robot, achieving equivalent task success rates (e.g., 100% for simple pick-and-place) as rigid manipulators after finetuning. This adaptation enabled safe, robust human-robot interaction and demonstrated resilience to physical perturbations, marking a step towards deploying intelligent AI in compliant robotic systems.
Key insights
- Out-of-the-box VLA policies trained on rigid robots fail completely when deployed on soft robots, primarily due to the significant kinematic and dynamic discrepancies.
- Targeted finetuning with a relatively small custom dataset can effectively bridge the embodiment gap, enabling VLA models to achieve performance on soft robots comparable to rigid counterparts.
- The choice of VLA architecture and finetuning strategy (e.g., LoRA with Llama 2 for OpenVLA-OFT) impacts adaptation effectiveness, with OpenVLA-OFT showing superior performance over "pi"_0 on the finetuned soft robot.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.