Reproduce “Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng, Sergey Levine, Yoshua Bengio
This paper introduces a modular reinforcement learning framework that replaces a centralized meta-policy with a decentralized information-theoretic competition mechanism among primitive policies. By making primitives pay an information cost for state access, the system encourages specialization and robust coordination, yielding improved generalization and transfer learning across various Minigrid, continuous control, and motion imitation tasks.
Key insights
- Forcing primitive policies to minimize the information they extract from the state through a KL divergence penalty naturally leads to their specialization in specific state regions or task aspects.
- Replacing a centralized meta-policy with a decentralized, information-driven competition mechanism among primitives eliminates a critical bottleneck, enabling better generalization and transfer learning.
- Distributing environment rewards to primitives based on their information demand and selection weight creates an incentive structure that balances performance with specialization and diversity among components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.