Reproduce “Collaborative Loco-Manipulation for Pick-and-Place Tasks with Dynamic Reward Curriculum” on OpenResearch

About this paper

Tianxu An, Flavio De Vincenti, Yuntao Ma, Marco Hutter, Stelian Coros

Researchers at ETH Zurich developed a hierarchical reinforcement learning framework enabling single and collaborative legged robots with arms to perform end-to-end pick-and-place tasks. This system, utilizing a dynamic reward curriculum, achieved high success rates in real-world experiments and improved training efficiency by 55% with 18.6% faster execution times in simulation for long-horizon mobile manipulation compared to existing multi-stage RL approaches.

Key insights

• The dynamic reward curriculum effectively mitigates learning bottlenecks, prevents catastrophic forgetting, and promotes the acquisition of recovery behaviors by adaptively scaling stage-wise rewards based on training progress.
• Hierarchical control enables seamless whole-body coordination, allowing a single policy to simultaneously manage complex locomotion and manipulation tasks for both individual and collaborative robots.
• Robots dynamically shift their attention to different environmental elements (e.g., own gripper, object handles, neighbor's base) during various task phases, demonstrating emergent coordination strategies like inferring a neighbor's motion through physical coupling.

Summary by alphaXiv.