Reproduce “SFMformer: A Spatial-Frequency Modulation Transformer for Lightweight Image Super-Resolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chih-Hsiang Yang, Chia-Min Lin, Ching-Yu Tsai, Yung-Che Wang, Jen-Shiun Chiang
SFMformer introduces a lightweight image super-resolution transformer that enhances sparse attention mechanisms by strategically integrating spatial feature extraction and wavelet-domain frequency modulation. The model achieves state-of-the-art performance on lightweight benchmarks, improving PSNR/SSIM scores across datasets while maintaining a parameter count under 1 million, and demonstrates practical deployment on a Raspberry Pi 5.
Key insights
- Sparse attention mechanisms exhibit distinct 'selection' and 'aggregation' stages, which can be separately targeted by specific feature enhancement modules for complementary performance gains.
- Spatial feature enhancement (DFE) improves the discriminability of tokens for the selection stage, while frequency-domain modulation (WMA) strengthens high-frequency components in the aggregated features.
- The gains from these spatial and frequency interventions can compound, demonstrating super-additive improvements on some benchmarks, suggesting they address different, non-overlapping constraints in the sparse attention pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.