Reproduce “Sat-JEPA-Diff: Bridging Self-Supervised Learning and Generative Diffusion for Remote Sensing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kursat Komurcu, Linas Petkevicius
Sat-JEPA-Diff separates satellite-image forecasting into two stages: predicting a future semantic representation with an IJEPA-style encoder and rendering that representation with a parameter-efficient Stable Diffusion 3.5 generator. On Sentinel-2 imagery, it reaches 0.8984 GSSIM and 0.1475 FID, preserving sharper geographic structures than deterministic baselines, though with lower PSNR and SSIM than PredRNN and unresolved concerns about dataset splitting and hallucinated details.
Key insights
- Forecasting in a semantic latent space can preserve structural gradients and boundaries better than directly regressing future pixels, even when pixel-level similarity is lower.
- The learned fusion of semantic features and coarse current-image layout allows diffusion to synthesize detail while retaining broad spatial organization; the reported gate settles near an even contribution from both inputs.
- The results illustrate a perception–distortion trade-off: Sat-JEPA-Diff improves GSSIM and FID, while PredRNN remains stronger on PSNR, SSIM, and L1 error.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.