Reproduce “Diversifying AI: Towards Creative Chess with AlphaZero” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tom Zahavy, Vivek Veeriah, Shaobo Hou, Kevin Waugh, Matthew Lai, Edouard Leurent, Nenad Tomasev, Lisa Schut and 2 more

AZ_db trains AlphaZero as a team of latent-conditioned players, then selects moves using search values, visit counts, and exploration bonuses. In held-out puzzle tests, fast Gap planning improves Hard Talkchess solve rate by 11.06 percentage points over player 0. The full configuration shows a 0.61-point Penrose decline, within reported error, while separate opening matches show +50.3 Elo versus AlphaZero.

Key insights

  • The primary challenge for AlphaZero on difficult chess puzzles is out-of-distribution generalization, rather than a fundamental lack of understanding.
  • A "diversity bonus" can be achieved in AI, where a team of diverse agents outperforms a single optimized agent, mirroring human collaborative problem-solving.
  • Intrinsically motivating agents to explore different behaviors during training leads to more robust and generalized AI systems.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.