Reproduce “Temporal Difference Learning for Diffusion Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Qizhen Ying, Yangchen Pan, Victor Adrian Prisacariu, Junfeng Wen

The method adds a temporal-difference penalty to diffusion and consistency-training losses, aligning changes in predicted posterior means across noise levels with diffusion’s change. In one-step sampling, TD+CT improves FID over CT on both 64×64 datasets, with FFHQ falling from 19.45 to 15.93. TD+EDM matches or improves EDM at 12–18 steps, within tests limited to 64×64 images and probability-flow ODE sampling.

Key insights

  • The diffusion denoising process can be effectively reinterpreted as a policy evaluation problem within a Markov Reward Process framework, enabling the application of TD learning principles.
  • Explicitly enforcing consistency of posterior-mean drifts across time steps is crucial for improving the temporal coherence and robustness of diffusion models, especially for few-step sampling.
  • Principled loss reweighting is essential for stable optimization of TD objectives in continuous-time diffusion models, as it addresses the varying magnitudes of gradients across different time steps.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.