Reproduce “Emergent Communication between Heterogeneous Visual Agents through Decentralized Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mikako Ochiai, Masatoshi Nagano, Tadahiro Taniguchi

This research demonstrates that heterogeneous visual agents can develop shared, visually informative communication systems through decentralized learning via a Metropolis–Hastings Captioning Game, without shared rewards or gradients. The study found that while increasing visual representation mismatch reduces overall communication performance, specific listener-side perceptual filtering is crucial for forming meaningful symbols, which can exhibit representational bias towards one agent's visual geometry.

Key insights

  • Meaningful, transferable communication can emerge between heterogeneous visual agents through decentralized learning and local perceptual evaluation alone, outperforming baselines that lack such interaction.
  • The degree of visual heterogeneity between agents directly impacts the quality, quantity, and specificity of emergent shared symbols, sometimes leading to a precision–coverage tradeoff.
  • Emergent communication systems can exhibit a representational bias, reflecting one agent's private visual geometry more than the other's, particularly under strong heterogeneity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.