Reproduce “RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuming Jiang, Siteng Huang, Shengke Xue, Yaxi Zhao, Jun Cen, Sicong Leng, Kehan Li, Jiayan Guo and 5 more
RynnVLA-001, developed by DAMO Academy, Alibaba Group, and Hupan Lab, presents a Vision-Language-Action (VLA) model that significantly improves robot manipulation by employing a two-stage pretraining curriculum on 12 million human ego-centric videos, achieving an average task success rate of 90.6% and outperforming state-of-the-art baselines like GR00T N1.5 (55.6%) and Pi0 (70.4%).
Key insights
- Large-scale human ego-centric video demonstrations can effectively mitigate robot data scarcity by providing rich manipulation priors for VLA models.
- A progressive pretraining curriculum, moving from general visual dynamics to explicit action understanding via human keypoint trajectories, is crucial for successful and robust skill transfer to robots.
- Utilizing an ActionVAE for action chunk representation leads to more efficient, compact, and temporally coherent action embeddings, improving robot control efficiency and smoothness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.