Reproduce “RL from Physical Feedback: Aligning Large Motion Models with Humanoid Control” on OpenResearch

About this paper

Junpeng Yue, Zepeng Wang, Yuxuan Wang, Weishuai Zeng, Jiangxing Wang, Xinrun Xu, Yu Zhang, Sipeng Zheng and 2 more

Researchers from Peking University and BeingBeyond introduced Reinforcement Learning from Physical Feedback (RLPF), a framework that fine-tunes large text-to-motion models with physical feedback from a motion tracking policy and semantic alignment verification. This approach generates physically feasible and semantically accurate motions for humanoid robots, achieving up to 0.97 success rate in IsaacGym and 0.75 in MuJoCo on the CMU dataset, outperforming baselines.

Key insights

• Direct physical feedback, obtained from a robust motion tracking policy, can effectively transform semantically accurate motions into physically feasible actions for robots.
• Semantic fidelity to text instructions during physical fine-tuning is preserved by integrating an alignment verification module.
• The motion tracking policy's architecture, such as the Exbody2-based approach, plays a key role in reducing the sim-to-real gap and enhancing real-world deployment.

Summary by alphaXiv.