Reproduce “Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georgy Perevozchikov, Nancy Mehta, Mahmoud Afifi, Radu Timofte
Rawformer, developed by researchers from the University of Würzburg and York University, presents an unsupervised Transformer-based method for raw-to-raw image translation between different camera models. It enables a single pre-trained learnable Image Signal Processor (ISP) to process raw images from diverse cameras, achieving superior fidelity with 41.89 PSNR on the NUS dataset and reducing the need for camera-specific training data.
Key insights
- Transformer architectures, when specialized with components like Condensed Query Attention and Scale Perceptive Feed-forward Networks, can accurately model the complex global and local transformations required for high-fidelity raw-to-raw translation.
- Robust unsupervised raw-to-raw translation effectively decouples learnable ISP development from camera-specific data acquisition, enabling a single pre-trained ISP to be universally applied across various camera models.
- Combining self-supervised pre-training (image inpainting) with a novel discriminator design (featuring CQA and a feature cache) significantly enhances training stability and performance in unsupervised domain adaptation for sensitive image data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.