Reproduce “SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Benjamin Ellis, Jonathan Cook, Skander Moalla, Mikayel Samvelyan, Mingfei Sun, Anuj Mahajan, Jakob N. Foerster, Shimon Whiteson

SMACv2 introduces a procedurally generated, more stochastic, and meaningfully partially observable benchmark for cooperative multi-agent reinforcement learning. This new benchmark addresses the original SMAC's limitations by preventing open-loop policy success and significantly challenging state-of-the-art algorithms, necessitating true generalization and robust coordination.

Key insights

  • The original SMAC's lack of stochasticity allowed agents to succeed by memorizing fixed action sequences or inferring Q-values from minimal observation, not requiring complex closed-loop policies.
  • Procedural generation and enhanced partial observability effectively force MARL algorithms to learn more generalizable and robust policies that truly condition on observations.
  • Random start positions emerged as a major factor in increasing SMACv2's difficulty, indicating the challenge of adapting to diverse initial spatial configurations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.