Reproduce “Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bradly C. Stadie, Sergey Levine, Pieter Abbeel
Augmenting rewards with a bonus derived from the misprediction error of a concurrently learned deep dynamics model enhances exploration in reinforcement learning, leading to faster and more consistent learning across various challenging Atari 2600 games compared to heuristic exploration methods.
Key insights
- The misprediction error of a learned dynamics model can serve as an effective, scalable proxy for novelty and uncertainty in high-dimensional state spaces.
- Learning meaningful low-dimensional state representations (via autoencoders) is crucial for building effective predictive models in complex visual environments.
- Dynamically updating the state representation and dynamics model alongside policy learning can yield better and more adaptive exploration incentives.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.