Reproduce “Accelerated Online Reinforcement Learning using Auxiliary Start State Distributions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aman Mehra, Alexandre Capone, Jeff Schneider
A hybrid reinforcement learning method, AuxSS, significantly improves the sample efficiency of online learning by dynamically selecting episode start states from limited expert demonstrations. It uses an approximation of "state safety" based on episode length to prioritize challenging states, achieving state-of-the-art performance on hard-exploration tasks and outperforming other hybrid methods while using 15x less expert data.
Key insights
- Leveraging a small amount of expert data and simulator resets can dramatically accelerate online reinforcement learning sample efficiency.
- Incorporating a notion of 'safety,' approximated via episode length, into the auxiliary start state selection is crucial for accelerated and robust training.
- Dynamic adaptation of the auxiliary start state distribution is essential, as static distributions or non-safety-informed approaches lead to inferior performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.