Reproduce “Pre-Trained Video Generative Models as World Simulators” on OpenResearch

About this paper

Haoran He, Yang Zhang, Liang Lin, Zhongwen Xu, Ling Pan

A framework called Dynamic World Simulation (DWS) enables pre-trained video generative models to function as controllable world simulators. This approach yields enhanced action-conditioned video prediction quality and increased efficiency in model-based reinforcement learning, demonstrating substantial performance improvements in various environments.

Key insights

• Existing pre-trained video generative models contain vast visual and dynamic priors that can be efficiently repurposed for interactive world simulation with minimal architectural modifications.
• Explicitly reinforcing dynamic changes through a weighted loss function is more effective for action-conditioned video prediction than uniform pixel-wise loss, leading to better temporal consistency.
• Prioritizing the imagination of trajectories from states where an agent exhibits higher learning potential significantly boosts sample efficiency and policy learning in Model-Based Reinforcement Learning.
• A lightweight, architecture-agnostic conditioning mechanism can universally imbue diverse video generative models with precise action control without extensive re-engineering.

Summary by alphaXiv.