Reproduce “SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, Zhijie Deng
SALA connects separately recorded human and robot videos through atomic-action descriptions, using graded semantic relationships to align human-video latents with robot latents grounded by robot-action reconstruction. Its policy learns from both video types with one action expert. On a 63-DoF robot, 2,678 hours of human-video pre-training reduced held-out action error by 2.21% versus none; increasing human demonstrations raised success by 18.75 percentage points in these tests.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.