Reproduce “CVPR 2023 Text Guided Video Editing Competition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jay Zhangjie Wu, Xiuyu Li, Difei Gao, Zhen Dong, Jinbin Bai, Aishani Singh, Xiaoyu Xiang, Youzeng Li and 12 more
The CVPR 2023 Text Guided Video Editing Competition established a new open-source benchmark dataset and evaluation framework for text-guided video editing. It identified Team CAMP's Two-Stage Video Editing (2SVE) method as the top performer based on human evaluation, while also revealing a significant discrepancy between automated metrics and human judgment.
Key insights
- A significant uncorrelation, and even anti-correlation, exists between automated metrics (e.g., CLIPScore, PickScore) and human evaluation results for generative video editing tasks.
- A two-stage finetuning approach combined with explicit strategies for consistency (VPA, MVSD) and precise control (TSP with ControlNet) is effective for high-quality, text-aligned, and temporally coherent video edits.
- Frame-by-frame editing, enhanced with mechanisms to leverage inter-frame consistency, conserves computational resources and supports higher resolution/arbitrary duration video editing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.