Reproduce “On the Topology, Distributions, and Dynamics of SGD and Weight Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Muyao Kong
The authors define an Ornstein–Uhlenbeck reverse diffusion for arbitrary Borel laws of SGD outputs. Starting from a standard Gaussian, its endpoint has exactly the target support at every positive horizon, while its distribution approaches the target in total variation as the horizon grows. The construction uses the target law’s score; it does not establish exact dynamic equivalence between SGD and denoising.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.