Reproduce “Physics-based Noise Modeling for Extreme Low-light Photography” on OpenResearch

About this paper

Kaixuan Wei, Ying Fu, Yinqiang Zheng, Jiaolong Yang

Researchers from Beijing Institute of Technology, The University of Tokyo, and Microsoft Research Asia introduced a comprehensive physics-based noise model for CMOS sensors, enabling the synthesis of highly realistic low-light training data for deep neural networks. This methodology allowed deep denoisers to achieve performance comparable to or exceeding models trained on real paired data, resolving the data scarcity issue and improving generalization across various camera devices and dynamic scenes.

Key insights

• Extreme low-light noise requires detailed physical modeling beyond simple Gaussian or Poisson distributions, incorporating observed long-tailed read noise distributions, significant color-wise DC biases, and systematic row noise patterns.
• High-fidelity synthetic training data, precisely generated from a physics-based model, can effectively substitute and even outperform real paired data for training deep denoisers, especially in terms of generalization to unseen camera devices.
• The ability to synthesize realistic dynamic noisy video data is crucial for training multi-frame video denoising networks that can effectively handle motion, addressing a significant challenge in low-light videography where real dynamic paired data is practically impossible to collect.

Summary by alphaXiv.