Reproduce “Lost Relatives of the Gumbel Trick” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matej Balog, Nilesh Tripuraneni, Zoubin Ghahramani, Adrian Weller
The authors transform random MAP results to estimate partition functions and sample discrete Gibbs distributions. Their exponential-distribution parameter-estimation approach extends Gumbel perturbations to derive low-rank bounds and sequential samplers. In Robust Bayesian Regression with A* sampling, the debiased Exponential estimator of ln Z used up to 40% fewer samples than Gumbel at equal MSE; low-rank results vary by setting.
Key insights
- The widely-used Gumbel trick for sampling and partition function estimation is a specific instance within a broader family of perturb-and-MAP methods, derivable by choosing different transformation functions.
- New members of this family, such as the 'Exponential trick', offer superior statistical efficiency (e.g., lower Mean Squared Error) for estimating partition functions compared to the Gumbel trick.
- These generalized tricks can be implemented with minimal computational overhead by post-processing the output of existing Gumbel-perturbed MAP solvers and are applicable to both full-rank and low-rank perturbation settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.