Reproduce “REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

George Tucker, Andriy Mnih, Chris J. Maddison, Dieterich Lawson, Jascha Sohl-Dickstein

REBAR reduces gradient-estimate variance in models with sampled discrete latent variables while keeping estimates unbiased for the discrete objective. It uses a relaxation as a control variate with corrections and optimized temperature. In a two-layer linear MNIST model its training variational lower bound was −98.8 versus −99.0 for MuProp over five initializations; this does not establish better test likelihood.

Key insights

  • An unbiased gradient estimator can be achieved by carefully constructing a control variate that leverages the low-variance properties of continuous relaxations.
  • Conditional marginalization plays a crucial role in enabling the derivation of a control variate that is both effective and maintains unbiasedness.
  • The unbiased nature of REBAR permits the relaxation temperature to be treated as a learnable parameter, allowing adaptive optimization to further reduce variance during training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.