Reproduce “Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bart van Marum, Aayam Shrestha, Helei Duan, Pranay Dugar, Jeremy Dao, Alan Fern

The Dynamic Robotics and Artificial Intelligence Laboratory at Oregon State University developed a quantitative and reproducible real-world benchmarking framework for humanoid standing and walking. They also proposed and evaluated a reinforcement learning controller trained with a minimally constraining reward function, which achieved perfect disturbance rejection and superior in-place rotation accuracy on the Digit humanoid robot compared to a clock-based RL controller and a manufacturer-supplied controller.

Key insights

  • Systematic, quantitative real-world benchmarking is essential for identifying subtle controller weaknesses and iteratively improving reinforcement learning policies for robotics.
  • Minimally constraining reward functions can enable reinforcement learning algorithms to discover more robust and adaptable behaviors for humanoid locomotion than highly prescriptive designs.
  • Quantitative evaluation reveals important performance trade-offs, such as learned policies excelling in disturbance rejection and rotational accuracy but potentially having lower energy efficiency compared to model-based controllers.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.