Reproduce “VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shangkun Sun, Xiaoyu Liang, Songlin Fan, Wenxu Gao, Wei Gao
This research from Peking University and Peng Cheng Laboratory introduces VE-Bench, a comprehensive benchmark for evaluating text-driven video editing quality. It provides VE-Bench DB, the first subjective-aligned dataset with human annotations, and VE-Bench QA, a novel human-aligned quantitative metric that significantly outperforms existing methods by evaluating text-video alignment, source-target relationship, and visual quality.
Key insights
- Objective metrics for text-driven video editing quality assessment often poorly correlate with human perception and provide incomplete evaluations.
- Comprehensive subjective evaluation requires considering text-video consistency, source-target fidelity, and overall edited video quality.
- Current video editing models show varying performance across editing types, with strengths in stylization and challenges in object removal or precise structural edits, highlighting the need for more advanced prompt comprehension and motion handling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.