Reproduce “Learning Multi-Agent Communication with Contrastive Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yat Long Lo, Biswa Sengupta, Jakob Foerster, Michael Noukhovitch

A framework called Communication Alignment Contrastive Learning (CACL) enables decentralized agents to learn an emergent, shared communication protocol by leveraging contrastive learning. This approach frames messages as distinct views of a common environment state, yielding superior task performance and increased protocol symmetry compared to state-of-the-art baselines in multi-agent environments.

Key insights

  • Framing inter-agent messages as multi-view data for a shared underlying state provides a principled and effective way to use contrastive learning for establishing common communication grounding in MARL.
  • Self-supervised contrastive learning can effectively induce highly symmetric and semantically consistent communication protocols in fully decentralized multi-agent systems, improving coordination.
  • Directly combining self-supervised communication objectives with reward-oriented gradients for the communication head can lead to optimization difficulties, suggesting distinct training pathways are beneficial.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.