Reproduce “RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haoyu Zhao, Xingyue Zhao, Zhian Su, Bohan Hou, Yinzhou Tang, Fengyi Wang, Kehan Li, Yuming Jiang and 6 more

RynnVLA-Latent uses video-derived visual transitions as shared training targets for robot manipulation, letting human, robot, and simulation videos train a policy before adaptation to a robot’s controls. RynnLAM separates motion from estimated camera change, but its descriptor retains a camera latent. After robot-specific post-training, its 4B policy scores 57.14% on steak serving, versus 40.00% for π⁰.⁵; robot demonstrations remain necessary.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.