Reproduce “Emergent Communication with Attention” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryokan Ri, Ryo Ueda, Jason Naradowsky
Integrating cross-modal attention mechanisms into artificial agents engaged in referential games leads to the emergence of communication protocols with increased compositionality and interpretability. Agents with attention in both the Speaker and Listener generally outperformed non-attention baselines in generalization accuracy and topographic similarity, demonstrating an ability to associate input attributes with specific symbols.
Key insights
- Attention mechanisms act as an effective inductive bias, promoting the development of more compositional and interpretable emergent languages in multi-agent systems.
- The benefits of attention are maximized when both Speaker and Listener agents are equipped with these mechanisms, suggesting a multiplicative effect in robust language emergence.
- Attention weights provide a verifiable link between emergent symbols and specific input attributes, revealing patterns of monosemy and semantically coherent polysemy in the learned vocabulary.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.