Reproduce “Nexus-VLA: Bridging Human and Robot Experiences via a Shared Hierarchical Semantic Latent Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Biaoliang Guan, Tingyu Yuan, Wen Ye, Ziming Ji, Xu Yang, Yan Huang, Jihua Zhu, Chaoyang Zhao
Nexus-VLA trains on unpaired human videos and robot demonstrations, shaping a shared latent with cues about intent, objects and locations, motion, and contact. Human wrist actions supervise shared end-effector controls, while robot data also supervises joint actions. On RoboCasa-GR1, it averages 80.5% across 24 tasks versus 72.8% for ACE-Ego-0; six unseen categories show 72.7% real-world success, not isolated human-data effects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.