Reproduce “Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hadi Alzayer, Philipp Henzler, Jonathan T. Barron, Jia-Bin Huang, Pratul P. Srinivasan, Dor Verbin
The method reconstructs 3D objects from photographs taken under different lighting by first jointly relighting the views to match a chosen input image, then fitting a reflection-aware radiance field. Per-image shading embeddings adjust normals to accommodate residual inconsistencies. On Objaverse’s shiny-asset split, it achieves 26.54 PSNR versus 23.46 for IllumiNeRF, for held-out views under the selected reference illumination.
Key insights
- Processing multiple views simultaneously within a diffusion model effectively disambiguates material properties from illumination changes, leading to more consistent relit images for subsequent 3D reconstruction.
- Per-image shading embeddings, which perturb surface normals, are crucial for resolving subtle errors from the relighting process without mistakenly absorbing view-dependent specularities into diffuse appearance, unlike standard appearance embeddings.
- Training the multiview relighting diffusion model on datasets augmented with purely specular objects significantly enhances its ability to learn robust priors about light transport, benefiting both highly shiny and standard assets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.