Reproduce “PatchVSR: Breaking Video Diffusion Resolution Limits with Patch-wise Video Super-Resolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shian Du, Menghan Xia, Chang Liu, Xintao Wang, Jing Wang, Pengfei Wan, Di Zhang, Xiangyang Ji
PatchVSR enables pre-trained video diffusion models to perform high-resolution video super-resolution by employing a patch-wise processing strategy with a dual-branch conditional guidance system. This approach allows 4K video generation from models trained at 512x512, yielding superior perceptual quality and reducing inference time by 2x and memory usage by over 1.5x compared to existing diffusion-based methods.
Key insights
- VSR primarily benefits from localized detail generation based on neighborhood semantics rather than intensive global coherence, suggesting patch-wise processing is a feasible and efficient strategy.
- Combining local patch conditions with global video context is essential to bridge the 'generation gap' and provide sufficient semantic guidance when adapting full-frame generative models to patch-level tasks.
- A training-free multi-diffusion strategy with weighted blending of overlapping patches effectively mitigates boundary artifacts and ensures visual and temporal consistency in patch-wise generative tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.