Reproduce “MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhenhan Yin, Xuanhan Wang, Jiahao Jiang, Kaiyuan Deng, Pengqi Chen, Shuangle Li, Chong Liu, Xing Xu and 3 more
MiVLA introduces a human-robot mutual imitation pre-training framework that enables Vision-Language-Action (VLA) models to generalize across tasks and embodiments without extensive real-world robot data. The approach achieved a 69% success rate on a challenging simulation benchmark and demonstrated competitive real-world performance, including bimanual manipulation on an unseen robot morphology.
Key insights
- Mutual imitation pre-training, where a model predicts actions for both the source and target embodiment, is crucial for effectively unifying disparate data sources and enabling robust cross-embodiment transfer.
- Bidirectional kinematic mapping provides a scalable way to convert between human and robot action spaces, overcoming the 'Sim2Real gap' and 'morphological discrepancies'.
- Combining behavior priors from simulated robots with real-world action knowledge from human videos can create generalizable robot policies without direct reliance on large-scale real robot pre-training data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.