Reproduce “MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shili Zhou, Xuhao Jiang, Weimin Tan, Ruian He, Bo Yan
MVFlow augments RAFT-style optical-flow estimation with motion vectors extracted directly from compressed video. A credibility-aware conversion module turns block-level codec motion into a dense initialization, improving robustness to strong H.264 compression and reducing the refinement iterations needed for comparable accuracy; at KITTI 2015 QP 37, it reaches 9.43 AEPE versus 11.33 for retrained RAFT, while a four-iteration setting cuts runtime by about 52%.
Key insights
- Motion-vector metadata is useful under compression only after representation conversion; directly using raw block vectors can degrade performance rather than improve it.
- The credibility-weighted local aggregation in MVCM allows image content to correct codec estimates while extending motion information into regions without valid vectors.
- Compression-specific training is itself important, but combining it with transformed motion priors yields further gains in accuracy and can reduce the number of recurrent updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.