Reproduce “GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuhao Liao, Peizhuo Li, Xinrong Yang, Linnan Chang, Zhaoxin Fan, Qing Wang, Lei Shi, Yuhong Cao and 2 more
Growing Policy Optimization (GPO) introduces a framework for training reinforcement learning policies for legged robots by dynamically adjusting the effective action space during training. This method enables faster and more stable learning, achieving 100% success rates in hardware robustness tests against external disturbances and challenging terrains, outperforming PPO and other baselines.
Key insights
- Dynamically evolving the effective action space through a smooth, time-dependent transformation preserves core RL update rules like PPO while bounding and diminishing gradient distortion.
- Restricting the action space in early training significantly reduces policy-gradient variance and improves the signal-to-noise ratio, leading to more stable and faster acquisition of low-amplitude control behaviors.
- An asymmetric growth rate in the action space, exemplified by the Gompertz curve, is more effective than linear or sigmoid functions for balancing stable initial learning and robust asymptotic performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.