Reproduce “RoboJEPA: Scaling Robotic Latent World Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Artem Zholus, Nicolas Beltran-Velez, Jianhao Yuan, Sarath Chandar, Tushar Nagarajan, Daniel Severo, Koustuv Sinha, Michal Drozdzal and 4 more
RoboJEPA predicts future visual features under robot actions and uses imagined outcomes to plan toward a goal image. Its predictor scales from 22 million to 8 billion parameters while keeping the V-JEPA 2.1 encoder fixed. A second-order compute law extrapolates held-out prediction error better than a standard power law, but does not establish a universal link to robot success.
Key insights
- A second-order power law forecasts held-out prediction error more accurately than a standard power law when extrapolating to 4B- and 8B-parameter models.
- Planning capabilities appear at different compute scales: basic reaching comes first, followed by grasped-object transport, obstacle avoidance, and object pushing.
- Longer-horizon tasks benefit from training with longer autoregressive rollouts, underscoring the importance of matching training to deployment demands.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.