Reproduce “FSCM: Frequency-Enhanced Spatial-Spectral Coupled Mamba for Infrared Hyperspectral Image Colorization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tingting Liu, Yuan Liu, Guiping Chen, Xiubao Sui, Qian Chen

Researchers from Nanjing University of Science and Technology and collaborators developed FSCM, a spectral-information-guided conditional Generative Adversarial Network, for colorizing infrared hyperspectral images into visible-light counterparts. The method utilizes a frequency-enhanced spatial-spectral Mamba generator to efficiently model long-range dependencies while recovering weak textures and refining local geometric structures, achieving improved visual fidelity and semantic consistency across diverse datasets.

Key insights

  • The integration of state-space models for efficient global dependency modeling with explicit frequency-domain enhancement (FEM) and local geometric refinement (DGM) effectively addresses the challenges of recovering fine details in infrared hyperspectral data.
  • A multi-path Mamba block design that couples global spatial, local spatial, and spectral processing allows for comprehensive utilization of high-dimensional hyperspectral information, leading to more coherent and semantically rich colorizations.
  • Incorporating an online semantic segmentation-guided loss provides crucial structural supervision, helping to mitigate boundary drift, prevent semantic confusion, and improve the retention of small objects in complex scenes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.