Reproduce “Diagnosing JEPA World Models with Action-Conditioned Predictive Consistency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guo An, Zijing Wu, Honghua Dong, Yuhao Yan, Zixuan Gui, Haochong Chen, Shanzhao Ruan, Xiang Wang and 2 more
Researchers introduce Action-Conditioned Predictive Consistency (ACPC) and related diagnostics, Invariance Radius (IR) and Separation Rate (SR), to evaluate the robustness of Joint-Embedding Predictive Architectures (JEPAs) in world models. The framework quantifies how visual perturbations propagate through latent dynamics and demonstrates that augmented models exhibit lower sensitivity to visual shifts while preserving essential state distinctions, providing a reliable method for identifying robust checkpoints.
Key insights
- ACPC offers a comprehensive, multi-step, action-conditioned diagnostic for world models, revealing how visual perturbations propagate through latent dynamics, which single-step metrics cannot capture.
- The combined IR-SR diagnostic effectively identifies world model checkpoints that are simultaneously robust to visual perturbations (low IR) and maintain necessary state discriminability (high SR).
- Theoretical derivations prove ACPC's relevance to model performance by bounding changes in prediction error and guaranteeing the stability of planning decisions under visual shifts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.