Reproduce “Back to Basics: Let Denoising Generative Models Denoise” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianhong Li, Kaiming He
JiT makes pixel-space diffusion predict clean images directly from noisy raw-image patches, then derives the sampling update from that estimate. A plain Transformer is trained with velocity-space squared error, motivated by a proposed lower-dimensional structure in clean images. On ImageNet 256×256, JiT-B/16 reaches FID-50K 8.62 versus 372.38 for noise prediction, but this contrast narrows at 64×64.
Key insights
- Predicting on-manifold clean data ("x") is more robust and inherently easier for neural networks than predicting off-manifold quantities (noise or "v") in high-dimensional input spaces.
- The "x-prediction" strategy effectively mitigates the "curse of dimensionality" for ViT-based pixel-space diffusion models, allowing them to function stably on raw, high-dimensional inputs.
- A simplified, self-contained "Diffusion + Transformer" framework, operating solely on pixels, can achieve competitive image generation performance without requiring complex external components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.