Reproduce “Learning Modular Robot Control Policies” on OpenResearch

About this paper

Julian Whitman, Matthew Travers, Howie Choset

Carnegie Mellon University researchers developed a scalable control framework for modular robots, leveraging Graph Neural Networks and Model-Based Reinforcement Learning to enable a single learned policy to command diverse robot designs. The approach successfully achieves zero-shot generalization to unseen configurations and demonstrates sim-to-real transfer to physical hardware.

Key insights

• Matching the GNN's structure to the robot's kinematic graph provides an effective inductive bias, enabling parameter sharing and generalization across diverse modular designs.
• Message passing within the GNN allows individual modules to adapt their behavior contextually, realizing "plug and adaptive play" control where a module's function depends on its neighbors and position.
• Learning from multiple robot designs simultaneously with shared GNN parameters for both dynamics and policy models allows for efficient knowledge transfer and generalization.

Summary by alphaXiv.