Reproduce “JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jie Xu, Kangjin Yu, Ziyi Jin, Junjie Gao, Liqing Chen, Yixian Li, Shuai Tian, Zhongpu Xia
JEPA Policy introduces a diffusion-free visuomotor imitation learning framework that enhances robot control by jointly predicting expert actions and their resulting future observation representations within a shared Transformer stack. This approach achieved an average 5.6 percentage point increase in simulation success rate over an action-only baseline and a 33x faster inference latency than Diffusion Policy, with strong performance validated on real robotic hardware.
Key insights
- The performance gains stem critically from the deep interaction and shared parameterization of action and future tokens within a single Transformer stack, rather than merely from an auxiliary future prediction loss.
- Expert action supervision serves as an inherent, task-driven anti-collapse mechanism for the shared representation, removing the need for explicit regularization in future-predictive learning.
- The policy's predicted future observation representation can be used as a low-cost, deployment-time diagnostic signal (future-consistency error) to identify potential policy failures and anomalous behaviors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.