Reproduce “Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matthew Thomas Jackson, Minqi Jiang, Jack Parker-Holder, Risto Vuorio, Chris Lu, Gregory Farquhar, Shimon Whiteson, Jakob Nicolaus Foerster
GROOVE, developed by researchers from the University of Oxford, UCL, and Google DeepMind, integrates Unsupervised Environment Design with meta-learning to discover general reinforcement learning optimizers. This approach achieves superior zero-shot generalization from simple Grid-World environments to complex Atari games by automatically generating diverse and informative meta-training curricula based on a novel Algorithmic Regret metric.
Key insights
- The characteristics of the meta-training distribution, specifically task diversity and informativeness, are crucial for the generalization performance of meta-learned reinforcement learning algorithms.
- Algorithmic Regret (AR), which benchmarks against a fixed, general RL algorithm, effectively identifies informative environments for training meta-learned optimizers, outperforming other UED metrics for this purpose.
- Using a non-optimal, general algorithm as an antagonist for AR provides a more practical and effective curriculum than benchmarking against an optimal policy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.