Reproduce “MoMaps: Semantics-Aware Scene Motion Generation with Motion Maps” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahui Lei, Kyle Genova, George Kopanas, Noah Snavely, Leonidas Guibas
MoMaps predict dense 3D motion from a single reference image by assigning each visible pixel a future path and generating the resulting motion maps with an image diffusion model. On BRIDGE, MoMap diffusion scores 0.7212 on quantized direction accuracy, versus 0.6856 for a same-representation VAE baseline, using best-of-10 samples. The small test set and single-view coverage limit the evidence.
Key insights
- Representing complex 4D motion as pixel-aligned Motion Maps enables the application of powerful 2D image generative models for 3D motion synthesis.
- Pre-trained 2D diffusion models can be effectively adapted and fine-tuned for high-fidelity 3D motion generation, efficiently leveraging existing image priors.
- An explicit 3D motion generation pipeline can provide a strong inductive bias for 2D video synthesis, leading to more physically consistent and controllable video outputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.