Reproduce “DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junfeng Li, Junjie He, Zhide Zhong, Yangyang Zheng, Pingyue Sheng, Jiayu Dong, Ruixin Li, Haodong Yan and 7 more
DyPES-VLA introduces a framework for learning a single generalist policy capable of controlling various robot embodiments by leveraging shared dynamics priors from visual data and embodiment-specific action generation. This approach enabled state-of-the-art or comparable performance across multiple simulation benchmarks and achieved 75.6% average success on real-world tasks, outperforming independent and other generalist policies.
Key insights
- Explicitly learning shared dynamics priors from diverse visual data (including action-free videos) significantly enriches the shared representation, enhancing cross-embodiment transfer and policy robustness.
- Decoupling shared knowledge (dynamics priors) from embodiment-specific control allows a single generalist policy to effectively manage heterogeneous robot morphologies and native action spaces.
- A modular action head with shared attention layers and statically routed, embodiment-specific experts provides an efficient way to handle distinct kinematic constraints and control semantics within a unified architecture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.