Reproduce “Collaborative Evolutionary Reinforcement Learning” on OpenResearch

About this paper

Shauharda Khadka, Somdeb Majumdar, Tarek Nassar, Zach Dwiel, Evren Tumer, Santiago Miret, Yinyin Liu, Kagan Tumer

Collaborative Evolutionary Reinforcement Learning (CERL), developed by researchers at Intel AI Lab and Oregon State University, integrates a portfolio of gradient-based DRL learners with a neuroevolutionary algorithm and a dynamic resource manager to address DRL's exploration and hyperparameter sensitivity challenges. This framework successfully solves the Mujoco Humanoid benchmark, a complex task where individual state-of-the-art DRL algorithms failed.

Key insights

• The synergistic combination of diverse gradient-based learners and a gradient-free evolutionary algorithm, coordinated by adaptive resource allocation, can solve complex tasks that individual components cannot.
• Varying key hyperparameters like the discount rate among a portfolio of learners naturally fosters diverse exploration without requiring explicit, engineered exploration mechanisms.
• A shared replay buffer significantly boosts sample efficiency by allowing all DRL learners and the evolutionary population to collectively benefit from the entire framework's generated experiences.

Summary by alphaXiv.