Reproduce “Assessing Evolutionary Terrain Generation Methods for Curriculum Reinforcement Learning” on OpenResearch

About this paper

David Howard, Josh Kannemeyer, Davide Dolcetti, Humphrey Munn, Nicole Robinson

This research systematically assesses evolutionary terrain generation methods and representation-agnostic feature descriptors for curriculum reinforcement learning. It demonstrates that Compositional Pattern Producing Networks (CPPNs) excel at generating diverse, challenging terrains, and that a curriculum combining multiple generators can accelerate agent learning, especially in early stages.

Key insights

• CPPNs are exceptionally effective at generating highly diverse and challenging terrains, leading to the highest coverage in the MAP-Elites archives.
• Combining terrains from multiple generators into a single curriculum significantly accelerates the agent's learning speed, especially during initial and intermediate training stages.
• MAP-Elites-based curricula, organized by representation-agnostic feature descriptors, dramatically outperform traditional single-feature-based curriculum learning strategies.
• Roughness, TPI, and TRI are reliable, representation-agnostic feature descriptors that correlate strongly with terrain difficulty for locomotion tasks.

Summary by alphaXiv.