Reproduce “Solving Imperfect-Information Games via Discounted Regret Minimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Noam Brown, Tuomas Sandholm

Researchers from Carnegie Mellon University introduced novel Counterfactual Regret Minimization (CFR) variants featuring various discounting schemes, with the DCFR_3/2,0,2 algorithm consistently outperforming the prior state-of-the-art CFR+ in diverse imperfect-information game scenarios. This approach achieved a 2-3x speedup in exploitability reduction for certain poker subgames, demonstrating more efficient strategic learning.

Key insights

  • Discounting regrets from earlier iterations reduces the persistent influence of large, initial mistakes, allowing algorithms to adapt more quickly and improve convergence in complex games.
  • The specific parameterization DCFR_3/2,0,2 (with positive regret discounted by t^3/2, negative regrets floored at zero, and average strategy weighted by t^2) offers a robust and broadly superior performance profile.
  • Discounting techniques, particularly LCFR, can substantially enhance the effectiveness of Monte Carlo CFR (MCCFR), making it more viable for large-scale imperfect-information games where full tree traversal is impractical.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.