Reproduce “Causal Policy Gradient for Whole-Body Mobile Manipulation” on OpenResearch

About this paper

Jiaheng Hu, Peter Stone, Roberto Martín-Martín

A reinforcement learning framework called Causal MoMa automatically discovers causal dependencies between robot actions and reward terms in whole-body mobile manipulation tasks. This framework then leverages these dependencies within a causal policy gradient to reduce variance, yielding policies that demonstrate successful zero-shot transfer from simulation to real-world mobile manipulators and improved performance over baselines.

Key insights

• Mobile manipulation tasks inherently possess a causal structure where specific robot action dimensions directly influence particular reward function terms.
• Leveraging these precise causal relationships between actions and rewards can significantly reduce the variance of policy gradient estimators in reinforcement learning.
• These causal dependencies can be automatically discovered from robot interaction data, eliminating the need for manual, human-engineered factorization.

Summary by alphaXiv.