Reproduce “Video4Edit: Viewing Image Editing as a Degenerate Temporal Process” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaofan Li, Yanpeng Sun, Chenming Wu, Fan Duan, YuAn Wang, Weihao Bo, Yumeng Zhang, Dingkang Liang
Baidu Inc. researchers introduce Video4Edit, a framework that recasts instruction-driven image editing as a degenerate temporal process, leveraging pre-trained video generation models. This approach achieves high-fidelity image editing, matching or surpassing leading baselines, while requiring only about 1% of the training data typically used by mainstream methods.
Key insights
- Rich 'single-frame evolution priors' embedded in video-pretrained models can be effectively transferred to image editing tasks, significantly reducing data dependency.
- Distilling knowledge through a combined objective function, including block-wise DiT distillation and tail-frame latent supervision, is crucial for stabilizing student output and preserving edit locality.
- Explicitly verbalizing the transition via 'evolution captions' provides vital semantic alignment for the video-pretrained teacher, leading to more coherent intermediate states and better distillation targets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.