Reproduce “Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition” on OpenResearch

About this paper

Jonah Siekmann, Yesh Godse, Alan Fern, Jonathan Hurst

Researchers at Oregon State University developed a periodic reward composition framework for deep reinforcement learning that enables the bipedal robot Cassie to learn and execute all common bipedal gaits. This method allows for smooth transitions between gaits, and the learned policies transferred successfully to the physical robot for robust locomotion in diverse outdoor environments.

Key insights

• A probabilistic periodic reward composition framework offers a principled and intuitive method for defining complex, dynamic, and cyclical robot behaviors without relying on external motion capture data or pre-defined trajectories.
• By parameterizing gaits based on intrinsic periodic properties like swing/stance phases and their relative timings, a single deep reinforcement learning policy can be trained to perform and smoothly transition between diverse locomotion behaviors.
• Dynamics randomization combined with specific behavior-activated penalties (e.g., transition penalties) is crucial for robust sim-to-real transfer and for ensuring a multi-gait policy learns desired, cohesive behaviors rather than undesirable hybrids.

Summary by alphaXiv.