Reproduce “Evolutionary and Coevolutionary Multi-Agent Design Choices and Dynamics” on OpenResearch

About this paper

Erik Hemberg, Eric Liu, Lucille Fuller, Stephen Moskal, Una-May O'Reilly

Researchers at MIT CSAIL systematically evaluated multi-agent design choices and learning dynamics for autonomous cyber agents within the CybORG simulation environment, demonstrating that grammar-based representations significantly outperform FSM-like approaches and exploring a novel LLM-supported mutation operator for generating agent code. The study revealed that competitive coevolution leads to more balanced, fluctuating performance compared to higher, but less robust, peaks achieved against static adversaries.

Key insights

• Competitive coevolution, while leading to lower peak performance compared to one-sided learning against a static adversary, results in more robust, adaptively balanced, and fluctuating performance for both sides.
• Grammar-based agent representations, which generate expressive Python code, achieved orders of magnitude higher training fitness than simpler FSM-like representations for complex cyber tasks.
• Integrating LLMs for mutation in Grammatical Evolution is feasible and affects performance, but its effectiveness is highly sensitive to prompt engineering and the specific LLM model used.

Summary by alphaXiv.