Reproduce “Adversarial Policies Beat Superhuman Go AIs” on OpenResearch

About this paper

Tony T. Wang, Adam Gleave, Tom Tseng, Kellin Pelrine, Nora Belrose, Joseph Miller, Michael D. Dennis, Yawen Duan and 3 more

Researchers from UC Berkeley, FAR AI, MIT, and McGill revealed that state-of-the-art superhuman Go AIs, like KataGo, are vulnerable to adversarial policies that exploit strategic blind spots, such as misjudging "cyclic groups" of stones. These adversaries achieved up to a 95.7% win rate against KataGo with 4096 MCTS visits and 72% against KataGo with 10 million MCTS visits, despite playing poorly against human amateurs.

Key insights

• Superhuman capabilities and extensive self-play training do not guarantee worst-case robustness, as state-of-the-art Go AIs are highly vulnerable to specific strategic exploits that lead to catastrophic failures.
• Increasing MCTS search depth is insufficient as a standalone defense, as even at computationally prohibitive search budgets (10 million visits), KataGo remained highly vulnerable to exploitation.
• The discovered exploits represent comprehensible strategic vulnerabilities, not subtle numerical perturbations, as a human Go expert was able to learn and manually execute the attack to beat superhuman AIs.

Summary by alphaXiv.