Reproduce “Future Video Synthesis with Object Motion Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yue Wu, Rongrong Gao, Jaesik Park, Qifeng Chen
A two-stage "separate-predict-composite" framework generates future video by decoupling static backgrounds from dynamic objects, predicting their motions via optical flow and rigid affine transformations respectively, then compositing them. The method achieves superior perceptual quality on Cityscapes and outperforms existing state-of-the-art methods on KITTI, effectively preserving object rigidity and reducing visual blur in long-term predictions.
Key insights
- Explicitly decomposing a scene into independently moving objects and a background allows for more accurate and structured motion prediction.
- Modeling dynamic object motion using rigid affine transformations effectively preserves object shape and prevents distortion in future frames.
- Predicting future optical flow for the background in a batch (multi-frame) manner, rather than recursively, mitigates error accumulation and blurriness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.