Reproduce “MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Teng Hu, Mingchun Lu, Yating Wang, Jiangning Zhang, Jinkun Hao, Ye Pan, Ran Yi, Lizhuang Ma and 1 more
A framework called MetaWorld allows multi-agent video world models to synthesize physically synchronized and identity-consistent egocentric video streams from single-view video data for open-domain environments. The method achieves high cross-view consistency, with a CVC score of 0.8454 in full-3D scenes, while maintaining identity fidelity and structural synchronization across multiple camera branches.
Key insights
- Multi-agent physical dynamics and synchronized motion data can be effectively extracted from readily available single-view egocentric videos, overcoming the data scarcity bottleneck for training.
- Explicitly decomposing an egocentric video into its camera motion, multi-subject trajectories, and identity references enables the generation of consistent multi-agent views from monocular input.
- Synchronizing the denoising process across parallel generation branches via inter-branch cross-attention is crucial for maintaining both static geometric and dynamic motion consistency in multi-agent video world models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.