Reproduce “Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Runhao Zeng, Dingjie Zhou, Qiwei Liang, Junlin Liu, Hui Li, Changxin Huang, Jianqiang Li, Xiping Hu and 1 more

Video2Reward introduces a framework that generates and refines reward functions for legged robot behavior learning by leveraging video demonstrations and large language models. The method achieved a 37.6% improvement in human-normalized score on Anymal tasks and 11.1% on Humanoid tasks compared to the Eureka baseline, while also producing behaviors with lower Dynamic Time Warping scores that closely mimic target videos.

Key insights

  • Visual data, in the form of keypoint trajectories extracted from videos, offers a more precise and controllable input for defining complex robot behaviors than abstract natural language, enabling LLMs to generate more effective reward functions.
  • An iterative feedback loop, grounded in objective visual comparison of learned robot movements against target videos, is critical for refining LLM-generated reward functions to achieve high behavioral fidelity and naturalness.
  • The successful integration of multimodal AI (computer vision for motion features, LLMs for code generation) with reinforcement learning provides a powerful approach to overcome traditional challenges in robot behavior learning, making it more accessible and efficient.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.