Reproduce “Anti-efficient encoding in emergent communication” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, Marco Baroni
Researchers at Facebook AI Research and affiliated institutions investigated whether Zipf's Law of Abbreviation (ZLA) spontaneously emerges in neural network communication systems. They found that agents developed 'anti-efficient' codes where frequent inputs were associated with longer messages, a phenomenon attributed to the listener's inherent preference for easily discriminable long messages when no speaker effort cost is present.
Key insights
- Neural networks in a simple signaling game, without explicit brevity pressure, naturally develop "anti-efficient" communication codes where more frequent inputs are encoded with longer messages.
- This anti-efficient encoding is primarily driven by the Listener network's intrinsic bias: longer messages are inherently easier for the listener to discriminate, and in the absence of a countervailing cost for the Speaker, the system optimizes for this discriminability.
- Zipf's Law of Abbreviation only emerges when an explicit "least effort" pressure (a message length penalty) is introduced for the Speaker, empirically demonstrating that ZLA arises from a delicate balance between speaker effort and listener discriminability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.