Reproduce “Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuangyu Han, Abhronil Sengupta
The authors train policy and value networks with equilibrium propagation for simulated quadruped locomotion, adapting PPO in a two-stage CPG–RES controller. Networks update from changes between relaxed states before and after an output nudge. In MuJoCo, walking performance is broadly comparable to backpropagation-trained PPO on several measures; stage-one gradient estimation uses about 4.3× less GPU memory, not power; hardware was not tested.
Key insights
- Equilibrium Propagation can effectively train high-dimensional continuous-control policies and value functions within a PPO framework for complex embodied robotics tasks.
- Stabilizing EP-based continuous control training requires specific mechanisms, including two-sided PPO ratio clipping and scaling the objective gradient with 1/\sigma for the log-standard deviation update.
- Observation preprocessing, such as Inverse Discrete Cosine Transform (iDCT), accelerates convergence to equilibrium within EP networks, reducing the required relaxation steps and improving stability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.