Reproduce “MAME : Model-Agnostic Meta-Exploration” on OpenResearch

About this paper

Swaminathan Gurumurthy, Sumit Kumar, Katia Sycara

Researchers at Carnegie Mellon University developed Model-Agnostic Meta-Exploration (MAME), a meta-reinforcement learning framework that explicitly separates exploration and exploitation policies and utilizes self-supervised learning for efficient task adaptation. MAME attained competitive or superior performance on continuous control benchmarks and achieved stable peak performance in sparse-reward environments, requiring fewer adaptation trajectories for effective learning.

Key insights

• Separating exploration from exploitation policies allows for more effective information gathering in novel environments and robust adaptation.
• Utilizing stable self-supervised or supervised regression objectives for inner-loop adaptation improves training stability and speed compared to traditional policy gradient methods.
• The choice of self-supervision signal is critical for effective adaptation, with N-step return prediction being particularly beneficial for tasks involving varying reward functions.

Summary by alphaXiv.