Reproduce “Multi-View Latent Action Pretraining for Cross-Embodiment Robot Learning from Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Oliver Hausdörfer, Felix Minzenmay, Mani Majd, Maximilian P. Christof, Shiyuan Zhang, Angela P. Schoellig
The authors train robot policies from action-free video by combining synchronized camera views to infer latent actions, compact signals of frame-to-frame motion, and using these signals to train a policy alongside robot demonstrations. Across MimicGen, LIBERO, and LIBERO-Plus, multi-view pretraining achieves 56.2% mean rollout success, versus 48.9% for single-view and 42.7% for action-only training, under the reported low-action-budget setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.