Reproduce “Latent Shadows: The Gaussian-Discrete Duality in Masked Diffusion” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guinan Chen, Xunpeng Huang, Ying Sun, Shijin Wang, Yanyong Zhang, Chao Wang

Researchers from multiple institutions introduce Masked Diffusion Duality and Scalar Trajectory Locking, enabling Masked Consistency Distillation (MCD) for discrete diffusion models. This framework achieves a 16x inference speedup and improves generative perplexity to 39.73 at 32 sampling steps, surpassing the unaccelerated teacher model and existing distillation baselines.

Key insights

  • Asymmetric masked discrete diffusion, unlike uniform discrete diffusion, can be precisely aligned with a continuous Gaussian process through a deterministic projection, extending the concept of diffusion duality.
  • A single, per-token scalar uniform noise variable is sufficient to determine the entire discrete masking trajectory, significantly simplifying the construction of coupled states for distillation.
  • A hybrid consistency objective, combining KL divergence for mutually masked regions and cross-entropy for ground-truth supervision in revealed regions, is critical for stable and high-quality distillation in the discrete domain.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.