Reproduce “Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State” on OpenResearch

About this paper

Tamim Zoabi, Ameen Ali, Lior Wolf

The authors introduce H-JEPA, a system for planning from camera images that separates a broad perceptual code from the projected control state scored by the planner. A Bures–Wasserstein prior shapes the code, while a phase-conditioned port-Hamiltonian predictor and tied port-inverse consistency structure action-sensitive prediction. On OGB-Cube, success is 91.93% versus Delta-JEPA’s 79.27%, within simulated offline pixel-based control and short-horizon planning.

Key insights

• The port-inverse consistency loss is mathematically equivalent to reweighting rollout prediction error in the directions where actions enter the dynamics, rather than training a separate action decoder.
• The planner’s useful latent state need not map cleanly to a small set of interpretable physical variables; some planning-relevant directions are not captured by standard physical-variable probes.
• The appropriate control-state dimension varies by task, so the projection rank is a planning-capacity choice rather than an estimate of the environment’s physical state dimension.

Summary by alphaXiv.