Reproduce “FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chieh-Hsin Lai, Yuhta Takida, Naoki Murata, Toshimitsu Uesaka, Yuki Mitsufuji, Stefano Ermon
Researchers from Sony AI and Stanford University developed FP-Diffusion, a regularization method for score-based diffusion models that enforces consistency with the newly derived score Fokker-Planck Equation (FPE). This approach significantly improved density estimation, reducing Negative Log-Likelihood (NLL) on MNIST VE from 3.73 to 3.30 bits/dim, and enhanced the conservativity of learned score functions.
Key insights
- Ground truth score functions adhere to a fundamental mathematical self-consistency property, the score Fokker-Planck Equation (FPE), which existing pre-trained models empirically violate.
- Explicitly enforcing the score FPE during training implicitly enhances the conservativity of learned score functions and improves density estimation by reducing KL divergence.
- Bounding the score FPE residual is theoretically linked to bounding errors in higher-order score matching, implying a more comprehensive consistency property.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.