Reproduce “HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba” on OpenResearch

About this paper

Yinuo Wang, Yuanyang Qi, Jinzhao Zhou, Pengxiang Meng, Xiaowen Tao

HuMam, an end-to-end deep reinforcement learning framework, integrates a Mamba encoder for humanoid locomotion control, demonstrating enhanced training stability, improved sample efficiency, and reduced actuation costs. This approach increased peak task performance by 5.8%, required 42.5% fewer samples to achieve a high return threshold, and reduced average joint torques by 9.6% compared to a feedforward neural network baseline.

Key insights

• A Mamba encoder can effectively process and fuse multimodal observations in a state-centric manner for humanoid locomotion, capturing structured dependencies with computational efficiency compared to traditional networks or heavier sequence models.
• Integrating a Mamba encoder promotes energy-efficient control through smoother representations and enhances robustness to observation noise and overall training stability.
• A well-structured multi-term reward function combined with a hierarchical control scheme involving low-gain PD controllers is crucial for achieving balanced and energy-efficient bipedal gaits in complex humanoid systems.

Summary by alphaXiv.