Reproduce “Safe Model-based Reinforcement Learning with Stability Guarantees” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Felix Berkenkamp, Matteo Turchetta, Angela P. Schoellig, Andreas Krause
This research introduces a model-based reinforcement learning algorithm that safely optimizes control policies for continuous state-action systems by integrating Gaussian process models with Lyapunov stability theory. The methodology enables safe exploration and iteratively expands the certified region of attraction, successfully demonstrated on a simulated inverted pendulum where the system never became unstable during learning.
Key insights
- Lyapunov stability can be formally verified with high probability for systems with unknown dynamics modeled statistically by imposing stricter decrease conditions on a discretized state space.
- Safe exploration in continuous state-action spaces is achievable by restricting data collection to regions where transitions are provably stable, ensuring the system remains within the estimated region of attraction.
- Simultaneous policy optimization and expansion of the certified safe operating region is possible through iterative learning, enabling adaptive control without compromising stability guarantees.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.