Reproduce “Stochastic Regret Minimization in Extensive-Form Games” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Gabriele Farina, Christian Kroer, Tuomas Sandholm

This paper presents a general framework that modularizes stochastic regret minimization in extensive-form games, allowing for the independent selection of regret minimization algorithms and gradient estimators. The work provides an exponentially improved high-probability regret bound for MCCFR and demonstrates that algorithms like FTRL can outperform MCCFR when paired with specific gradient estimation methods, such as external sampling.

Key insights

  • The proposed framework successfully decouples regret minimization algorithms from gradient estimation methods in extensive-form games, enabling modular algorithm design.
  • A dramatically improved theoretical high-probability regret bound for Monte Carlo Counterfactual Regret Minimization (MCCFR) was derived, from O(√(T/p)) to O(√(T log(1/p))).
  • The choice of gradient estimator significantly impacts algorithmic performance, with external sampling generally favoring Follow-the-Regularized-Leader (FTRL) and balanced outcome sampling favoring MCCFR.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.