Reproduce “LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zicheng Duan, Jiatong Xia, Zeyu Zhang, Wenbo Zhang, Gengze Zhou, Chenhui Gou, Yefei He, Feng Chen and 2 more
LiveWorld introduces a framework that addresses the "out-of-sight dynamics" problem in generative video world models by decoupling the autonomous evolution of the world from observation rendering. This approach allows dynamic entities to continue progressing even when unobserved, ensuring consistent and naturally evolving environments. The system demonstrates superior performance on the new LiveBench benchmark, successfully tracking and updating out-of-sight dynamics, including a 26% "Full Success" rate for concurrent events where previous methods failed.
Key insights
- The conflation of view-independent world evolution with view-dependent observation rendering in prior models is a fundamental limitation, leading to the "out-of-sight dynamics" problem.
- Decomposing the world state into a static background and dynamic entities, and managing their independent, continuous evolution via monitors, is crucial for persistent world simulation.
- Utilizing a unified state-conditioned video backbone with explicit geometric guidance and complementary appearance references effectively generates consistent out-of-sight dynamics and coherent re-observations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.