Reproduce “Context-Aware Safe Reinforcement Learning for Non-Stationary Environments” on OpenResearch

About this paper

Baiming Chen, Zuxin Liu, Jiacheng Zhu, Mengdi Xu, Wenhao Ding, Ding Zhao

Researchers at Carnegie Mellon University introduce Context-Aware Safe Reinforcement Learning (CASRL), a framework enabling reinforcement learning agents to adapt safely and quickly in non-stationary environments. It combines model-based meta-learning with explicit safety constraints and prior domain knowledge, demonstrating improved safety violation rates and robust performance on robotic control tasks.

Key insights

• Safety and non-stationarity are intertwined challenges in real-world reinforcement learning, where dynamic disturbances directly influence safety boundaries.
• Integrating prior domain knowledge through a pre-trained model and a "prior safety constraint" can significantly enhance safety during early adaptation in uncertain environments.
• Explicitly modeling context and uncertainty with Attentive Neural Processes enables fast and robust adaptation to unseen environmental changes, even in high-dimensional tasks.

Summary by alphaXiv.