Reproduce “MV-Forcing: Long Multi-View Video Generation via 4D-Grounded Spatio-Temporal Self-Forcing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gal Fiebelman, Hadar Averbuch-Elor, Sagie Benaim
MV-Forcing generates long multi-view videos by sequentially extending time and camera views around a persistent CUT3R reconstruction, which provides geometric priors and confidence maps to a causal diffusion student. For three views and 162 frames, it reaches 239.37K matching pixels versus 146.81K for SF+ReCamMaster. The framework remains bounded by quality degradation over long horizons.
Key insights
- Unifying long temporal horizons with multi-view geometric consistency is achievable by combining autoregression along both temporal and view dimensions, avoiding the quadratic scaling of bidirectional attention.
- A persistent, incrementally updated 4D geometric prior, derived from an autoregressive 3D reconstruction model, can effectively provide continuous spatial grounding for sequential multi-view generation.
- Extending self-forcing with view-sequential unrolling during training is crucial to mitigate exposure bias when generating views conditioned on prior imperfect outputs, enabling robust and scalable multi-view autoregression.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.