Reproduce “SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yongchao Huang
SJEPA is a Joint-Embedding Predictive Architecture (JEPA) that learns predictive latent states whose dynamics are described by compact, explicit symbolic laws, complemented by a neural correction. This framework balances representation compression with operator compression to identify the "simplest adequate dynamics," allowing for more interpretable world models.
Key insights
- The complexity of learned dynamics is heavily dependent on the chosen latent coordinate system, highlighting the need for joint optimization of representations and their governing equations.
- Minimizing dynamics complexity without enforcing informative representations can lead to a 'collapse shortcut,' where the latent space degenerates to trivial fixed points, making dynamics appear artificially simple.
- Regularizing the neural correction component is crucial for a principled decomposition, encouraging the symbolic part to explain as much of the dynamics as possible while the neural part accounts for true residuals.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.