Reproduce “Oops I Took A Gradient: Scalable Sampling for Discrete Distributions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, Chris J. Maddison

GWG uses gradients of differentiable functions underlying discrete models to guide local proposals, then applies a Metropolis–Hastings correction to preserve the target distribution. In tested RBMs, it tracks a structured Block-Gibbs sampling reference as dimension grows. For static MNIST, its trained energy-based model reports −80.01 test log-likelihood, an AIS upper-bound estimate, not a definitive comparison with prior-work VAE.

Key insights

  • Many discrete probability mass functions are implicitly implemented as differentiable functions, allowing gradients to serve as a universally available and efficient form of local information for discrete sampling.
  • Approximating local likelihood ratios with gradients provides a computational speed-up (e.g., O(D) for single-variable changes) while theoretical bounds guarantee only a constant factor loss in efficiency under smoothness assumptions.
  • Operating directly in discrete space with gradient-informed proposals avoids the challenges and tuning complexities inherent in continuous relaxation-based gradient sampling methods.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.