Reproduce “TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuhui Chen, Haoran Li, Zhennan Jiang, Haowei Wen, Dongbin Zhao

The TeViR framework enables reinforcement learning agents to learn complex robotic manipulation tasks efficiently by generating dense, continuous rewards from raw pixel observations and natural language instructions. It utilizes pre-trained text-to-video diffusion models to create long-horizon 'expert' trajectories, achieving superior sample efficiency and stability compared to sparse reward and other diffusion-based baselines across various simulation and real-world tasks, even without ground truth environmental rewards.

Key insights

  • Dense rewards derived from long-horizon video generation by diffusion models provide a stable and informative signal, significantly improving sample efficiency and learning stability in RL.
  • The TeViR framework can effectively guide RL agents to learn policies even in the complete absence of ground truth environmental rewards, demonstrating autonomy in reward generation.
  • TeViR's reinforcement learning foundation allows it to adapt and learn even from suboptimal or imperfectly generated videos, showing robustness compared to methods that directly derive policies from generated frames.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.