Reproduce “GeoFlow: Enforcing Implicit Geometric Consistency in Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jan Ackermann, Shengqu Cai, Boyang Deng, Zhengfei Kuang, Songyou Peng, Gordon Wetzstein
GeoFlow fine-tunes a video generator with an explicit geometric-consistency reward, combining camera-motion and background structure with appearance similarity along motion trajectories. Rewards come from pretrained depth and optical-flow predictors, not 3D or 4D ground truth. On dynamic scenes with simple prompts, its multi-view consistency error is 5.638, versus 12.292 for Wan2.1-1.3B, in the tested evaluation rather than for all videos.
Key insights
- Geometric consistency in videos can be directly optimized using a dense, self-supervised reward signal derived from readily available monocular depth, optical flow, and feature predictors.
- Decoupling consistency into rigid background motion (structural) and independent object appearance preservation (semantic) effectively addresses general dynamic scenes with both camera and object movement.
- Strategic RL adaptations, including noise-synchronized latent initialization and truncated backpropagation, are crucial for making high-resolution video generation fine-tuning computationally tractable and stable.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.