Reproduce “Coordination Graphs for Constrained Multi-Agent Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Santiago Amaya-Corredor, Miguel Calvo-Fullana, Anders Jonsson
Researchers at Universitat Pompeu Fabra introduced CG-CMARL, a framework that integrates coordination graphs with Lagrangian duality to address scalability and safety constraints in multi-agent reinforcement learning. This approach successfully generates superior performance-safety Pareto fronts compared to baselines on cooperative navigation tasks with up to 10 agents, allowing for dynamic trade-off adjustments from a single trained model.
Key insights
- Decoupling the primary and cost Q-functions allows a single trained policy to dynamically adjust the objective-constraint trade-off at evaluation time without retraining.
- Max-Sum coordination enables scalability by efficiently solving the joint action selection problem in a decentralized manner.
- Lagrangian duality provides a principled, theoretically grounded approach to enforce safety constraints in multi-agent systems, converging to optimal dual solutions under specific conditions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.