Reproduce “Think While You Generate: Discrete Diffusion with Planned Denoising” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sulin Liu, Juno Nam, Andrew Campbell, Hannes Stärk, Yilun Xu, Tommi Jaakkola, Rafael Gómez-Bombarelli

DDPD pairs a corruption planner with a replacement denoiser for discrete diffusion. It adaptively revisits positions judged corrupted. On ImageNet 256×256 discrete-token generation, at 32 steps without logit annealing, DDPD records FID 4.75, versus 8.09 for MaskGIT and 29.12 for standard mask diffusion. Exceptions at 8 steps and with annealing limit the result to tested settings.

Key insights

  • Explicitly learning a 'planner' to identify corrupted positions significantly improves generative quality and robustness in discrete diffusion models.
  • An adaptive Gillespie sampler, guided by the planner and incorporating time correction, enhances sampling efficiency and enables continuous error correction during generation.
  • Decomposing the generative process into distinct planner and denoiser components simplifies training objectives derived from the ELBO, allowing for modularity and potential reuse of pretrained denoisers.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.