Reproduce “Edge-Efficient Image Restoration: Transformer Distillation into State-Space Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Srinivas Soumitri Miriyala, Sowmya Vajrala, Sravanth Kodavanti, Vikram Nelvoy Rajendiran, Sharan Kumar Allur

Researchers at Samsung Research Institute Bangalore developed a modular hybrid image restoration framework that combines distilled state-space model blocks with transformers, enabling an efficient network search for edge devices. This approach led to up to 3.4x faster inference on commercial mobile chipsets while preserving or enhancing restoration quality across deblurring, denoising, and deraining tasks.

Key insights

  • There is a notable discrepancy between theoretical computational efficiency (FLOPs) and actual inference latency on edge hardware, necessitating hardware-aware architectural optimization.
  • Feature-wise knowledge distillation effectively enables MambaIR blocks to function as surrogates for transformer blocks, expanding their receptive field and mimicking contextual understanding.
  • An efficient network search can identify hybrid transformer-SSM architectures that achieve a superior balance of image restoration quality and edge device latency compared to models relying solely on one architecture type.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.