Reproduce “Thermal is Always Wild: Characterizing and Addressing Challenges in Thermal-Only Novel View Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
M. Kerem Aydin, Vishwanath Saragadam, Emma Alexander
A novel view synthesis pipeline was developed to overcome the unique challenges of thermal-only imagery, such as low dynamic range and radiometric drift, by employing a contrast-stabilizing preprocessor and an adapted 3D Gaussian Splatting model with appearance embeddings. This approach achieves state-of-the-art performance in thermal 3D reconstruction, showing an average PSNR of 26.14 dB and SSIM of 0.88 across multiple datasets, while eliminating common visual artifacts.
Key insights
- Rigorous analysis of multiview thermal datasets identified consistent obstacles for thermal-only NVS: low dynamic range, significant radiometric instability, and attenuated spatial-frequency content.
- Incorporating appearance embeddings into the emission model of 3DGS is crucial for effectively managing and absorbing residual frame-dependent radiometric fluctuations, which are characteristic of thermal imagery.
- A synergistic effect occurs where photometric preprocessing stabilizes inputs and provides stronger optimization gradients, significantly benefiting the subsequent emission modeling within the 3DGS framework.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.