Reproduce “Multi-Agent Generative Adversarial Imitation Learning” on OpenResearch

About this paper

Jiaming Song, Hongyu Ren, Dorsa Sadigh, Stefano Ermon

The Multi-Agent Generative Adversarial Imitation Learning (MAGAIL) framework extends adversarial imitation learning to multi-agent Markov games, enabling agents to learn complex behaviors from demonstrations without explicit reward functions. This approach significantly outperforms baselines in cooperative and competitive tasks and demonstrates improved adaptability in environments with imperfect expert data.

Key insights

• Maximum entropy Inverse Reinforcement Learning can be extended to multi-agent Markov games by formulating it as finding reward functions that induce Nash equilibria matching expert occupancy measures.
• Adversarial training effectively bypasses the explicit recovery of reward functions in multi-agent settings, similar to GAIL in single-agent contexts, by implicitly learning a reward that distinguishes expert from generated behavior.
• Integrating prior knowledge about agent reward structures (e.g., shared rewards for cooperation) significantly improves MAGAIL's performance and relevance across various multi-agent interaction types.

Summary by alphaXiv.