Reproduce “Masked Autoregressive Flow for Density Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
George Papamakarios, Theo Pavlakou, Iain Murray
MAF improves flexible density estimation by stacking autoregressive layers that model the internal random numbers produced by earlier layers, while retaining tractable likelihood evaluation. On unconditional CIFAR-10, five-layer MAF reports average test log likelihood of 2,936 ± 27 nats, versus 2,576 ± 27 for five-layer Real NVP. This supports density estimation, not fast sampling or better sample quality.
Key insights
- An autoregressive model with Gaussian conditionals can be formally interpreted as an invertible transformation suitable for normalizing flows, with an easily computable Jacobian determinant.
- Training a MAF model by maximizing likelihood is equivalent to variationally training an implicit Inverse Autoregressive Flow (IAF) where MAF's base distribution acts as its target.
- Real NVP's coupling layers are a special case of MAF's more general autoregressive transformations, indicating MAF's superior flexibility and efficient use of model capacity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.