Reproduce “Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning” on OpenResearch

About this paper

Runsheng Yu, Zhenyu Shi, Xinrun Wang, Rundong Wang, Buhong Liu, Xinwen Hou, Hanjiang Lai, Bo An

Researchers at Nanyang Technological University and Sun Yat-sen University developed a multi-agent deep reinforcement learning framework that adapts regret minimization to foster cooperation and employs a differential particle filter for robust state estimation in partially observable, non-stationary environments. This approach consistently outperformed several baseline methods across various cooperative tasks, including large-scale battle games with 256 agents, achieving higher average total rewards and demonstrating improved scalability.

Key insights

• Regret minimization, typically used in competitive game theory, can be effectively reformulated and applied at a team level to induce cooperation in multi-agent systems.
• Decomposing team-level regret into individual agent regrets through additive or shaping functions is crucial for enabling decentralized execution while maintaining a globally cooperative policy.
• Integrating a differentiable particle filter allows for robust and accurate state inference from partial observations, significantly improving agent decision-making in highly uncertain environments.

Summary by alphaXiv.