Reproduce “BYOL-Explore: Exploration by Bootstrapped Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhaohan Daniel Guo, Shantanu Thakoor, Miruna Pîslar, Bernardo Avila Pires, Florent Altché, Corentin Tallec, Alaa Saade, Daniele Calandriello and 6 more
BYOL-Explore introduces a self-supervised method that unifies representation learning, world dynamics modeling, and curiosity-driven exploration within reinforcement learning. This approach enables agents to solve challenging 3D environments like DM-HARD-8 without requiring human demonstrations, and it achieves superhuman performance on the 10 hardest Atari exploration games with a simpler design than previous state-of-the-art agents.
Key insights
- A single, unified self-supervised latent-predictive loss can simultaneously learn robust representations, model world dynamics, and drive effective curiosity-based exploration.
- The bootstrapping mechanism, where the target network is an Exponential Moving Average of the online network, provides stable and non-trivial learning signals crucial for effective world model learning and exploration.
- Multi-step latent prediction is vital for efficient exploration in complex 3D environments, while one-step prediction can be sufficient for 2D, fully-observable settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.