Reproduce “Mastering the Game of Go with Self-play Experience Replay” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingbin Liu, Xuechun Wang
A model-free reinforcement learning algorithm named QZero achieved a 5th Dan level in Go through self-play, demonstrating the capability of value-based off-policy methods in complex environments without Monte-Carlo Tree Search. The system accomplished this using only 7 GPUs, significantly less than previous MCTS-based Go AIs, and learned sophisticated game concepts from scratch.
Key insights
- Model-free reinforcement learning can efficiently master the complex game of Go, demonstrating performance comparable to early AlphaGo versions with a purely learning-based approach.
- Value-based off-policy reinforcement learning is robust enough for large-scale, complex environments when augmented with stabilizing techniques like the Ignition Mechanism and entropy regularization.
- Achieving strong amateur Dan level Go AI is feasible with drastically fewer computational resources, specifically using only 7 GPUs compared to the thousands of TPUs employed by prior MCTS-based systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.