Reproduce “Critical windows: non-asymptotic theory for feature emergence in diffusion models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marvin Li, Sitan Chen
This research develops a formal, non-asymptotic theory explaining the phenomenon of "critical windows" in diffusion models, which are specific, narrow time intervals during reverse sampling when distinct features of generated outputs are determined. The work provides rigorous bounds for these windows and validates them empirically, demonstrating applications in model interpretability, fairness analysis, and membership inference attacks.
Key insights
- The "Master Theorem" formally proves the existence of critical windows in diffusion models, showing that within these specific time intervals, the noising-denoising process effectively shifts the distribution to a desired target sub-mixture.
- Explicit non-asymptotic bounds for critical times reveal that these windows emerge when the 'salience' of a target feature is high (intra-group separation is smaller than inter-group separation), with the window's length logarithmically dependent on this ratio.
- Diffusion models can be interpreted as 'hierarchical samplers,' progressively resolving output features through a discrete sequence of critical times that correspond to decisions on higher-level features before finer details.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.