Reproduce “On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Matthew King-Roskamp, Gabriel Rioux, Rustum Choksi, Tim Hoheisel

Researchers at McGill University and Imperial College London demonstrated that the empirical Maximum Entropy on the Mean (MEM) method achieves a parametric O(n^-1/2) convergence rate, improving upon previous bounds. They also developed a scalable Projected Clipped Stochastic Gradient Descent algorithm, enabling MEM to be applied to large-scale data-driven inverse problems by reformulating its dual problem for stochastic optimization.

Key insights

  • The empirical MEM estimator is statistically efficient, achieving an optimal parametric convergence rate of O(n^-1/2) for general convex and L-smooth fidelity terms.
  • The empirical MEM dual problem can be effectively re-expressed as an expected risk minimization problem, paving the way for scalable, data-driven applications using stochastic optimization.
  • Combining gradient clipping with a projection step is essential for stable and convergent optimization of the exponential loss functions characteristic of the reformulated MEM dual problem.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.