Reproduce “WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotations” on OpenResearch

About this paper

Xiaojie Xu, Zhengyuan Lin, Runyi Li, Yihao Liu, Kaipeng Zhang, Yongtao Ge

WorldRover is a scalable synthetic video data engine built on Unreal Engine 5.5 that generates richly annotated, long-range exploration videos from diverse virtual environments, providing ground-truth camera motion, scene geometry, temporal correspondence, and control signals. The associated WorldRover-10M dataset contains 21.9 million frames (202.7 hours) with multi-modal annotations, enabling controlled re-observation of trajectories from various viewpoints and environmental states.

Key insights

• Treating an "exploration" as a reusable world-space trajectory allows for generating consistent, multi-view observations and controlled re-rendering under varied environmental conditions, which is crucial for isolating experimental variables.
• Direct, renderer-derived ground-truth annotations (e.g., precise depth, amodal point tracks, distinct camera/character motion) provide explicit, high-fidelity supervision signals, enabling AI models to better disentangle visual changes and maintain coherent world representations.
• A robust, production-grade synthetic data generation pipeline, built on industry-standard engines like Unreal Engine, can scalably produce diverse, richly annotated datasets for complex AI research challenges.

Summary by alphaXiv.