Reproduce “MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tristan Tomilin, Luka van den Boogaard, Samuel Garcin, Constantin Ruhdorfer, Bram Grooten, Fabrice Kusters, Yali Du, Andreas Bulling and 2 more

The paper introduces MEAL, the first hardware-accelerated, JAX-based benchmark for Continual Multi-Agent Reinforcement Learning (CMARL), enabling efficient experimentation with long task sequences (e.g., 100 tasks in 4 hours on one GPU). Research using MEAL revealed that standard continual learning methods struggle with long-horizon forgetting and coordination breakdown in multi-agent settings, highlighting the need for specialized CMARL approaches.

Key insights

  • Hardware-accelerated benchmarks are crucial for CMARL, allowing 100-task sequences to run on a single GPU in hours, revealing long-horizon performance dynamics and failure modes not observable in short sequences.
  • Standard continual learning methods like Fine-Tuning suffer catastrophic forgetting, while regularization-based methods (e.g., PackNet, Online EWC) prevent forgetting but often incur negative forward transfer, and their effectiveness varies with the underlying MARL algorithm.
  • CMARL presents unique challenges beyond single-agent CL, including coordination forgetting (degradation of cooperative strategies despite maintained task performance) and the amplified difficulty of credit assignment and non-stationarity with more agents.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.