Reproduce “Neural Recursive Belief States in Multi-Agent Reinforcement Learning” on OpenResearch

About this paper

Pol Moreno, Edward Hughes, Kevin R. McKee, Bernardo Avila Pires, Théophane Weber

DeepMind researchers developed a scalable method for approximating hierarchical belief structures using neural recursive deep generative models in multi-agent reinforcement learning. This approach enables agents to form beliefs about other agents' knowledge and intentions, leading to improved performance in complex competitive and cooperative environments.

Key insights

• Approximating hierarchical beliefs with neural generative models provides a scalable framework for complex multi-agent reinforcement learning environments.
• Representing beliefs as nested collections of samples (rather than single samples) is crucial for accurately capturing the underlying belief distributions and achieving optimal performance in tasks requiring nuanced reasoning about others' knowledge.
• Integrating learned recursive belief states into an agent's policy input significantly enhances performance in multi-agent environments compared to model-free baselines.

Summary by alphaXiv.