Reproduce “SkipSR: Faster Super Resolution with Token Skipping” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rohan Choudhury, Shanchuan Lin, Jianyi Wang, Hao Chen, Qi Zhao, Feng Cheng, Lu Jiang, Kris Kitani and 1 more
SkipSR accelerates diffusion-based video super-resolution by selectively processing high-detail regions while entirely bypassing complex computations for low-detail areas. This framework achieves up to 60% end-to-end speedup for 720p videos and 70% reduction in diffusion time for 1080p videos, maintaining or improving perceptual quality compared to dense-attention models.
Key insights
- Not all regions in a video require the same computational intensity for super-resolution; a significant portion of low-detail patches can be processed cheaply without perceptual degradation.
- Completely skipping transformer computation for identified low-detail regions offers more substantial speedups than merely optimizing attention mechanisms within the transformer.
- Accurate prediction of 'skippable' regions from low-resolution input, combined with adapted positional encodings, enables efficient sparse processing while maintaining visual fidelity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.