Reproduce “Deep Sensorimotor Control by Imitating Predictive Models of Human Motion” on OpenResearch

About this paper

Himanshu Gaurav Singh, Pieter Abbeel, Jitendra Malik, Antonio Loquercio

A new method enables robots to learn complex sensorimotor skills by training a policy to imitate a predictive model of human motion. This approach, which uses the human motion predictor as a learned reward signal, allows robots to acquire dexterous manipulation abilities using sparse task rewards, matching performance typically achieved with meticulously engineered dense rewards and generalizing across different robot hands and tasks.

Key insights

• The growing similarity in form factor between humans and anthropomorphic robots allows human keypoint motions to serve as a robust, shared abstraction for learning.
• A predictive model of human motion, trained solely on human data, can be applied zero-shot to guide robot policy learning by providing an intelligent, behavior-shaping reward signal.
• Decoupling the human dataset from the robot policy learning loop offers a more scalable and robust approach compared to traditional kinematic retargeting or adversarial imitation methods.

Summary by alphaXiv.