Reproduce “WLA$^3$: World Latent Action Modeling for Semantics, Dynamics, and Kinematics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peidong Liu, Zhiyuan Xiang, Mingyang Li, Wenhao Li, Jiale Zhang, Jiahao Sun, Jiawei Li
WLA³ introduces a shared transition representation for learning robot policies from human video, robot trajectories, simulation, and wearable-interface demonstrations without requiring a common action space. Its World Latent Action Model combines multiview observations and embodiment state into compact latent actions, improving representation quality and raising six-task real-robot success to 81.9%, compared with 66.2% for π0.5.
Key insights
- A compact latent action can preserve both semantic and control-relevant information: the final 32-dimensional representation reaches 67.89% classification accuracy and 0.49 control-regression MSE on LARYBench.
- Multiview input and embodiment state improve transition understanding; adding wrist views and state raises classification accuracy over primary-view input, while cross-modal dropout and overlap regularization provide additional gains.
- Human videos are useful as transition supervision rather than direct action demonstrations: task-matched human data raises success on the Eraser task from 40% to 60% when robot data are reduced to 5%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.