Reproduce “Characterizing Speed Performance of Multi-Agent Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Samuel Wiggins, Yuan Meng, Rajgopal Kannan, Viktor Prasanna
This research provides a systematic analysis of Multi-Agent Reinforcement Learning (MARL) algorithm speed performance, introducing a taxonomy and demonstrating that communication overhead is a significant bottleneck on multi-core CPU platforms. The findings highlight latency-bounded throughput as a crucial performance metric, with empirical results showing varied communication costs across algorithms like MADDPG, ToM2C, and NeurComm.
Key insights
- Communication overhead is a non-trivial and often dominating factor in MARL system execution time, with its impact varying significantly based on the algorithm's training scheme and communication method.
- Learnt communication methods (e.g., ToM2C) generally incur higher computational costs, particularly in the Model Update phase due to complex graph neural network computations for message passing.
- Decentralized training schemes (e.g., NeurComm) exhibit a much higher reliance on communication, accounting for over 70% of agent execution time in some cases, highlighting challenges in the absence of global information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.