Reproduce “Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zihao He, Yunfeng Wu, Xinchao Wang, Songhua Liu
He et al. from Shanghai Jiao Tong University developed Flexible Image Transformer (FIT), an all-in-one image restoration model that incorporates degradation-aware deformable tokenization throughout its processing pipeline. The model improves restoration quality by adapting patch boundaries to local degradation severity, achieving up to 1.44 dB PSNR gain over previous methods on five diverse degradation tasks.
Key insights
- Fixed patch tokenization creates a fundamental bottleneck in Transformer-based image restoration by introducing grid artifacts and limiting adaptability to heterogeneous degradations.
- Extending degradation awareness to the entire restoration pipeline, including the initial image-to-token and final token-to-image interfaces, is crucial for mitigating artifacts and improving restoration quality.
- Dynamically adjusting patch boundaries based on local degradation information allows the model to align sampling with degradation structures, thereby enhancing detail recovery and spatial consistency without explicit supervision on offset patterns.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.