Reproduce “Simple Baselines for Image Restoration” on OpenResearch

About this paper

Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, Jian Sun

Researchers at MEGVII Technology propose simple, computationally efficient baselines, including a "Nonlinear Activation Free Network" (NAFNet), for image restoration. NAFNet achieved 40.30 dB PSNR on SIDD denoising, outperforming previous state-of-the-art with less than half the computational cost, and 33.69 dB PSNR on GoPro deblurring, using only 8.4% of the prior SOTA's Multiply-Adds, demonstrating that explicit non-linear activation functions are not necessary for high performance.

Key insights

• High performance in image restoration can be achieved with significantly simpler network architectures and block designs, challenging the trend of increasing model complexity.
• Explicit non-linear activation functions like ReLU or GELU are not strictly necessary for state-of-the-art deep learning models; inherent non-linearity from element-wise products (gating mechanisms) can be sufficient and even superior.
• Layer Normalization is crucial for stabilizing training and enabling higher learning rates in deep image restoration networks.

Summary by alphaXiv.