Reproduce “Generative Video Propagation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaoteng Liu, Tianyu Wang, Jui-Hsien Wang, Qing Liu, Zhifei Zhang, Joon-Young Lee, Yijun Li, Bei Yu and 3 more
Generative Video Propagation (GenProp), developed by researchers from CUHK, Adobe Research, and HKUST, presents a unified framework for diverse video editing tasks by consistently propagating modifications made to a single first frame across an entire video sequence. This approach achieved higher PSNR, CLIP-Text, and CLIP-Image scores on challenging video editing benchmarks, and was strongly preferred by users over competing baselines for visual quality and instruction alignment.
Key insights
- Complex video editing tasks can be effectively unified and simplified by framing them as a generative propagation problem from a single edited first frame.
- Designing specialized components like a Selective Content Encoder and employing a Region-Aware Loss is critical for balancing content preservation and generative modification.
- Synthetic data generation, including techniques like copy-and-paste and mask-and-fill, is a viable strategy to train comprehensive video editing models, addressing the scarcity of real-world paired video data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.