Reproduce “Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy” on OpenResearch

About this paper

Danica J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex Smola, Arthur Gretton

The authors develop optimized maximum mean discrepancy (MMD) tests that learn kernel parameters or representations by estimated test power rather than raw distance, with separate training and testing samples. On binarized MNIST, a learned pixel-weighted kernel reported p-values of 0.000 in 98 of 100 evaluations and 0.001 in two. This result does not isolate the power objective.

Key insights

• Explicitly maximizing the t-statistic for MMD test power leads to more sensitive and reliable statistical tests compared to common heuristic kernel selection methods.
• MMD, particularly with optimized kernel selection and feature matching, offers a principled and stable criterion for GAN training, which can effectively encourage sample diversity and mitigate mode collapse.
• Analyzing optimized kernels (e.g., ARD weights) and the witness function provides interpretable insights into the specific features or regions where generated and real data distributions differ, offering actionable feedback for model development.
• Significant computational efficiencies in MMD permutation tests can be achieved through optimized memory access and parallelization, expanding the practical applicability of MMD-based analyses to larger datasets.

Summary by alphaXiv.