Reproduce “Replay-Guided Adversarial Environment Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
This paper establishes a theoretical connection between adversarial environment design and prioritized level replay, proposing Dual Curriculum Design (DCD) as a unifying framework. The authors introduce new algorithms, PLR⊥ and REPAIRED, demonstrating their effectiveness in enhancing zero-shot generalization for reinforcement learning agents across diverse environments.
Key insights
- Prioritized Level Replay (PLR), traditionally an empirical strategy, can be theoretically grounded within an Unsupervised Environment Design (UED) framework.
- Training RL agents on a carefully curated, smaller set of replayed high-regret levels can lead to greater robustness and zero-shot generalization than training on more data from a generative process.
- Dual Curriculum Design (DCD) unifies replay-based level curation (like PLR) with adversarial level generation (like PAIRED) into a single theoretical framework.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.