Reproduce “Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning” on OpenResearch

About this paper

Aditya Kapoor, Yash Bhisikar, Benjamin Freed, Jan Peters, Mingfei Sun

This research generalizes the Differentiable Discrete Communication Learning (DDCL) framework, enabling multi-agent systems to dynamically control message precision for unbounded signals. The approach reduces communication bandwidth by one to five orders of magnitude while maintaining or improving task performance across various multi-agent reinforcement learning benchmarks and demonstrates that general-purpose architectures combined with learned communication efficiency can outperform specialized communication modules.

Key insights

• Agents learn efficient, frequency-aware communication protocols, allocating fewer bits to more frequent events (e.g., 0.25 bits for 51.5% frequent vs. 15.98 bits for 0.3% frequent).
• The communication cost coefficient (λ) effectively controls the rate-distortion trade-off, enabling agents to achieve lossless then lossy compression.
• A simple Transformer-based policy equipped with DDCL matched or exceeded the performance of complex, specialized communication architectures, supporting the 'Bitter Lesson' in MARL communication.

Summary by alphaXiv.