Reproduce “Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret” on OpenResearch

About this paper

Haitham Bou Ammar, Rasul Tutunov, Eric Eaton

Researchers at the University of Pennsylvania developed a lifelong policy gradient reinforcement learning algorithm that ensures adherence to safety constraints while achieving sublinear regret of O(sqrt(R)). The method consistently maintained policies within safe regions and significantly accelerated learning across sequential tasks in dynamical systems such as quadrotors, requiring only one observation to achieve a safe policy.

Key insights

• The proposed method achieves sublinear regret of O(sqrt(R)), demonstrating improved long-term learning efficiency and more effective knowledge accumulation compared to linear regret methods.
• Explicitly embedding safety constraints into the online policy search ensures that learned policies operate within acceptable boundaries, critical for real-world deployment.
• Representing task-specific policies through a shared latent knowledge base allows for efficient and safe knowledge transfer across a sequence of diverse tasks.

Summary by alphaXiv.