Reproduce “Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Experts” on OpenResearch

About this paper

Onur Celik, Aleksandar Taranovic, Gerhard Neumann

Di-SkilL introduces a reinforcement learning framework that acquires diverse, multi-modal skills for continuous control tasks by integrating a Mixture of Experts policy with Automatic Curriculum Learning, leveraging Energy-Based Models for per-expert context distributions. This approach achieved high success rates across various robotic simulation tasks, demonstrating the ability to learn multiple distinct solutions for the same task context, such as diverse table tennis strikes and reaching trajectories.

Key insights

• Energy-Based Models (EBMs) provide a robust and flexible way to parameterize per-expert context distributions for ACL, effectively representing complex, multi-modal, or discontinuous regions without needing explicit context bounds.
• Integrating automatic curriculum learning allows individual experts to specialize and efficiently learn distinct skills by focusing on specific sub-regions of the context space.
• The combination of a Maximum Entropy objective and stable trust-region updates enables the reliable acquisition of diverse, non-linear, multi-modal behaviors in continuous, contextual reinforcement learning tasks.

Summary by alphaXiv.