Reproduce “Demo-JEPA: Joint-Embedding Predictive Architecture for One-shot Cross-Embodiment Imitation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingyang He, Guangrun Li, Jieyu Zhang, Chengkai Hou, Zhengping Che, Shanghang Zhang
Demo-JEPA introduces a framework for one-shot cross-embodiment imitation learning by inferring target-compatible latent goals from visual demonstrations using a Joint Embedding Predictive Architecture. The approach allows robots to transfer skills across diverse morphologies, achieving a 0.36 success rate in zero-shot simulated tasks and 0.25 in real-world scenarios, outperforming concatenation baselines.
Key insights
- Decoupling the intent of a demonstration from its embodiment-specific execution, by interpreting demonstrations as future latent goals, enables more robust cross-embodiment skill transfer.
- Joint Embedding Predictive Architectures (JEPA) effectively learn semantically meaningful latent representations that abstract away nuisance visual variations and morphological differences, facilitating generalization.
- An adaptive goal updating mechanism, which advances the current subgoal only upon successful achievement, stabilizes long-horizon tasks and prevents premature temporal progression in planning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.