Reproduce “Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leonardo F. Toso, Yann LeCun, James Anderson, Oumayma Bounou
The authors introduce EP-IDM, which trains a world model to predict future latent states and reconstruct action sequences from endpoint latents. For linear systems, exact reconstruction under sufficient action excitation preserves information on reachable directions. In simulated CartPole, latent LQR success rises from 0% with SIGReg to 100% with EP-IDM; however, SIGReg models also succeed under nonlinear replanning.
Key insights
- For linear systems, latent feedback can stabilize the system only if the representation preserves observability of every unstable or marginally stable mode; prediction loss and SIGReg alone do not ensure this.
- Under the paper’s assumptions, exact endpoint action reconstruction makes the encoder injective on the finite-horizon reachable subspace, linking action supervision to retention of control-relevant directions.
- Benefits depend on the control setting and encoder: planners can sometimes compensate for model errors that defeat latent LQR, and a frozen visual encoder may already have discarded information EP-IDM needs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.