Reproduce “Seeing through Light and Darkness: Sensor-Physics Grounded Deblurring HDR NeRF from Single-Exposure Images and Events” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yunshan Qi, Lin Zhu, Nan Bao, Yifan Zhao, Jia Li

Researchers developed See-NeRF, a Neural Radiance Field (NeRF) framework that reconstructs sharp, High Dynamic Range (HDR) 3D scene representations from single-exposure blurry LDR images and corresponding event data. This approach explicitly models the distinct sensor-physics of both RGB and event cameras, leading to improved HDR and deblurring novel view synthesis performance on synthetic and real-world datasets.

Key insights

  • Explicitly modeling the distinct sensor-physics of both RGB cameras (via a 2D pixel-wise CRF) and event cameras (via a latency-aware, photometrically calibrated event CRF) is crucial for accurate HDR and deblurring NVS from combined single-exposure LDR images and events.
  • Applying the Camera Response Function at the pixel level after volume rendering, rather than at 3D points, is critical for accurately learning the CRF curve and achieving superior performance in HDR and deblurring NVS.
  • Event data provides vital complementary spatial and temporal dynamic information that significantly enhances both HDR and deblurring novel view synthesis, assisting the model in learning a more accurate RGB sensor CRF curve.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.