Reproduce “ROMA: Multi-Agent Reinforcement Learning with Emergent Roles” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tonghan Wang, Heng Dong, Victor Lesser, Chongjie Zhang

ROMA is a multi-agent reinforcement learning framework that enables agents to learn dynamic and specialized roles guided by two novel regularizers. This approach facilitates adaptive policy sharing and demonstrates superior performance in challenging cooperative tasks such as StarCraft II micromanagement.

Key insights

  • Roles can emerge dynamically through a learning process, allowing for flexible task decomposition and specialization without relying on prior domain knowledge.
  • The explicit enforcement of 'identifiable' and 'specialized' properties on emergent roles through mutual information-based regularizers is critical for effective multi-agent coordination and performance.
  • Adaptive policy sharing, mediated by dynamically learned roles, provides a superior balance between efficiency and specialization compared to monolithic shared policies or fully independent policies.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.