Reproduce “Learning the Target Priors Before Image Translation: A Decoupled Training Paradigm for Cross-Modal Image Translation in Remote Sensing” on OpenResearch

About this paper

Keyan Hu, Mingtao Wang, Ziyu Zhou, Tiandong Shi, Haifeng Li, Ji Qi, Chao Tao

Hu et al. introduce a decoupled training paradigm, Learning the Target Priors Before Image Translation (LTP-BIT), that first pretrains an unconditional generative prior for the target remote sensing domain before performing conditional adaptation. This approach achieves state-of-the-art performance across SAR-to-RGB and NIR-to-RGB translation tasks, operating with 9.81% of task-specific parameters and demonstrating improved data efficiency.

Key insights

• Decoupling the learning of target priors from cross-modal dependence allows for more efficient and robust model development by leveraging abundant unpaired target data for prior learning and focusing limited paired data on source-target correspondence.
• The utility of a target-domain prior for downstream translation is influenced not only by its scale but also by its local feature-structure compatibility and its coverage of the task-relevant feature manifold.
• Achieving high instance-level fidelity (e.g., PSNR) from stronger, more realistic generative priors requires a commensurate increase in the capacity of the conditional adaptation module.

Summary by alphaXiv.