Reproduce “ReDDiT: Rehashing Noise for Discrete Visual Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianren Ma, Xiaosong Zhang, Boyu Yang, Junlan Feng, Qixiang Ye

A new framework named ReDDiT enhances discrete visual generative models by introducing "rehashing noise" and a principled "rehash sampler". This approach allows discrete models to achieve state-of-the-art performance, with the ReDDiT-XLf8 model reaching a gFID of 1.61 on ImageNet-1K, outperforming leading continuous diffusion models.

Key insights

  • Expanding the discrete noise space from a single mask to multiple "rehashed" absorbing states significantly enhances the model's capacity to learn complex data distributions.
  • A principled sampler, directly derived from discrete diffusion theory, provides more stable and probabilistically faithful generation compared to heuristic re-masking approaches.
  • Integrating a Representation Alignment (RepA) loss, previously used in continuous models, proves effective for discrete diffusion models, accelerating training convergence and improving generation quality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.