Reproduce “Safe Reinforcement Learning with Model Uncertainty Estimates” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Björn Lütjens, Michael Everett, Jonathan P. How
A Safe Reinforcement Learning framework integrates computationally tractable model uncertainty estimates from MC-Dropout and bootstrapping, enabling autonomous agents to detect novel observations and navigate cautiously. This leads to fewer collisions in dynamic, safety-critical environments and facilitates more effective exploration during policy training.
Key insights
- Model uncertainty, measured by the variance of collision probabilities, effectively identifies novel or out-of-distribution observations in a regional manner, prompting cautious behavior.
- An adaptively increasing variance penalty (λ_v) during training enhances exploration by encouraging the agent to investigate uncertain regions, helping to escape local optima and achieve more robust policies.
- Integrating uncertainty estimates enables autonomous systems to exhibit more graceful and predictable cautious behavior in unforeseen circumstances, improving generalizability from simulation to reality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.