Reproduce “NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, Daniel Duckworth

Google Research developed NeRF-W, an extension of Neural Radiance Fields, capable of synthesizing novel views from unconstrained photo collections by modeling per-image appearance variations and handling transient objects. The method achieved an average PSNR improvement of 4.4dB over the previous state-of-the-art on Phototourism datasets, producing sharper and more accurate renderings.

Key insights

  • Disentangling photometric appearance from static scene geometry and modeling transient elements with an explicit uncertainty mechanism is essential for robust novel view synthesis from unconstrained data.
  • Latent appearance embeddings provide a continuous, controllable space for manipulating lighting and stylistic properties of a rendered scene without altering its underlying 3D structure.
  • An uncertainty-aware loss function, combined with a transient density regularizer, effectively enables the model to automatically ignore dynamic occluders and focus on reconstructing the permanent scene.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.