Reproduce “Jacobian-Aware Posterior Sampling for Inverse Problems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liav Hen, Tom Tirer, Raja Giryes, Shady Abu-Hussein
JAPS combines direct and proximal guidance for diffusion-based image restoration. It projects a measurement-fitting estimate onto the direction of a Jacobian-aware estimate, retaining prior-informed geometry while calibrating its magnitude. On ImageNet-256 random inpainting with 90% missing pixels and noise σᵧ = 0.05, JAPS reports 23.71 dB PSNR and 0.168 LPIPS, versus DDPG’s 21.85 dB and 0.395. This is a single-task comparison.
Key insights
- The denoiser's Jacobian acts as a crucial differentiator between direct and proximal likelihood approximations, encoding local geometric information of the data distribution.
- Trained denoisers exhibit non-ideal Jacobians (non-symmetric, non-positive semi-definite) which, if used directly, degrade performance in zero-shot inverse problem reconstruction.
- A projection-based fusion of direct and proximal likelihood estimates effectively leverages the valuable prior information within the Jacobian while mitigating the detrimental effects of its empirical non-idealities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.