Reproduce “LaLa-World: Scaling Latent World Models with Natural Language Actions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junha Song, Geonmo Gu, Jeongeun Park, Heejeong Nam, Lucas Maes, Byeongho Heo, Dongyoon Han, Sangdoo Yun
LaLa-World lets one latent world model plan across control domains by translating numerical actions and visual changes into language, then predicting future visual features with a language-model-based predictor. In simulator tests, its jointly trained model averages 72.1% success across five tasks, versus 64.6% for task-specific JEPA-WMs. These results do not establish real-robot policy deployment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.