Reproduce “LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bo Jiang, Depu Meng, Yihan Hu, Yichen Xie, Tianshuo Xu, Wei Zhan
LaMo learns motion cues from ordinary unlabeled videos by measuring changes between nearby latent frames. It uses a spatially averaged drift target during training and a learned spatial predictor to guide sampling. On VideoPhy, LaMo-5B scores 73.0/41.0 SA/PC, versus CogVideoX-5B at 70.0/32.3. These benchmark gains do not establish physical correctness; VBench improves in reported metrics.
Key insights
- Unlabeled video datasets inherently contain rich, self-supervised evidence of physical regularities and motion dynamics that can be effectively extracted and leveraged.
- Decomposing the latent motion prior into global (macro drift) and spatially resolved (micro field) components, with stage-matched integration, optimizes both training stability and inference performance.
- Applying motion guidance in the noise prediction space, rather than directly editing latents, prevents desynchronization and maintains a more coherent sampler trajectory.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.