Reproduce “DeepJEPA: Scaling World Models from Within” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zijian Jin, Yunbei Zhang, Yuanzhe Liu, Ming Liu, Baian Chen, Weirui Ye, Shilong Liu, Marco Pavone
DeepJEPA lets planners refine latent states within imagined transitions when useful, while keeping candidate-plan count and rollout horizon fixed. A continue head selects recurrent depth for each candidate and step, using latent prediction improvement as a proxy. On Cube Triple, it reaches 77.3% success versus 74.0% for the best fixed depth, at mean depth 1.22; this is not a runtime-speedup claim.
Key insights
- More uniform depth does not consistently help: depth four fails to outperform depth one across the five evaluated settings, while intermediate fixed-depth results are also non-monotonic.
- Refinement is concentrated around interactions: in PushT, transitions refined beyond depth one rise from 1.24% immediately before contact to 9.02% at contact onset.
- An update can matter through small changes in candidate costs that affect CEM elite selection, even without broadly improving object-state decodability; sufficiently small cost corrections leave the elite set and next proposal unchanged.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.