Reproduce “Behavioral Monitoring of JEPA World Models with Jacobian Centroids” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Thomas Walker, Randall Balestriero, Richard Baraniuk
The paper monitors world-model planning by comparing encoder and predictor behavior, using Jacobian centroids—the row-sums of input–output Jacobians—as a signal alongside latent activations. A pre-action gate flags low predictor alignment and can redirect selected episodes to closer goals. On TwoRoom, gated resampling reached 48.0 ± 7.3% success versus 38.4% for random redirection, across LeWorldModel, Push-T, and TwoRoom only.
Key insights
- Centroids expose a distinction between representational knowledge and behavioral responsiveness: an encoder may represent the target correctly even when the predictor is not locally oriented toward reaching it.
- Expert trajectories are smoother and more temporally organized in centroid space than shuffled controls, with centroid paths consistently showing greater straightness and lag-1 autocorrelation than activation paths.
- Predictor layer usefulness is task-dependent, and the strongest monitoring signal comes from identifying encoder-predictor dissociation rather than relying on a single universal layer or anomaly metric.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.