Reproduce “Emergent Social Learning via Multi-agent Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kamal Ndousse, Douglas Eck, Sergey Levine, Natasha Jaques
This research investigates emergent social learning in independent multi-agent reinforcement learning, demonstrating that agents augmented with an auxiliary state prediction loss can autonomously learn to utilize social cues from experts. The approach improves performance in complex exploration tasks and enhances zero-shot generalization to novel environments, even leading to performance exceeding that of the observed experts.
Key insights
- An auxiliary loss focused on predicting next states is critical for enabling independent RL agents to effectively extract and learn from implicit social cues in sparse-reward settings.
- Agents can develop opportunistic social learning strategies, such as 'free-riding' on expert exploration, which can lead to performance surpassing that of the experts they observe.
- Social learning can foster a generalized policy for online adaptation, enabling agents to robustly transfer skills to novel environments and unknown experts without retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.