Reproduce “Scalable Control Variates for Monte Carlo Methods via Stochastic Optimization” on OpenResearch

About this paper

Shijing Si, Chris. J. Oates, Andrew B. Duncan, Lawrence Carin, François-Xavier Briol

This research introduces a general framework for constructing and learning control variates (CVs) for Monte Carlo methods by leveraging Stein operators and stochastic optimization. The approach provides significant computational speed-ups and variance reduction across various synthetic and real-world Bayesian inference problems, making advanced CV techniques scalable to large datasets and high dimensions.

Key insights

• Stein operators offer a powerful and unified mechanism for generating a wide range of valid control variates, encompassing established and novel function classes.
• Stochastic optimization allows for the scalable and efficient training of complex control variates, transforming methods previously limited by cubic or exponential computational costs into practical tools for large-scale applications.
• Ensemble control variates, combining different function classes (e.g., kernels and polynomials), often yield superior variance reduction, demonstrating the flexibility and benefit of the unified framework.

Summary by alphaXiv.