Reproduce “Student of Games: A unified learning algorithm for both perfect and imperfect information games” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Martin Schmid, Matej Moravcik, Neil Burch, Rudolf Kadlec, Josh Davidson, Kevin Waugh, Nolan Bard, Finbarr Timbers and 5 more

Student of Games (SOG) introduces a unified learning algorithm that combines guided search, self-play, and game-theoretic reasoning, achieving strong empirical performance in both perfect information games like Chess and Go, and imperfect information games such as No-Limit Texas Hold'em and Scotland Yard. The algorithm provides theoretical guarantees for convergence to approximate Nash equilibria.

Key insights

  • A single unified framework can effectively combine deep learning, guided search, and game-theoretic reasoning to address complexities in both perfect and imperfect information games.
  • GT-CFR and continual re-solving provide theoretical soundness, guaranteeing that the algorithm's policies converge to an approximate Nash equilibrium with increasing computation.
  • Self-play, augmented by a novel bootstrapping method from 'recursive queries' within GT-CFR searches, efficiently generates diverse training data to improve the CVPN's generalization and value estimates.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.