Reproduce “Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, Pratul P. Srinivasan

Ref-NeRF enhances Neural Radiance Fields (NeRF) to accurately render specular and glossy surfaces by reparameterizing view-dependent appearance using reflection directions and an Integrated Directional Encoding. The method achieves a PSNR of 35.96 dB and a normal accuracy (MAE) of 18.380 on glossy datasets, outperforming mip-NeRF by 6.2 dB and over 40 degrees respectively, while also enabling intuitive scene editing.

Key insights

  • Reparameterizing view-dependent appearance using reflection directions, instead of viewing directions, simplifies the underlying function that neural networks must learn for glossy surfaces.
  • The Integrated Directional Encoding (IDE) allows for natural interpolation of view-dependent appearance across varying material roughness by attenuating high-frequency components for rougher materials.
  • Directly predicting surface normals combined with a novel 'orientation loss' encourages opaque, sharp surfaces and prevents NeRF's common issue of
  • foggy
  • geometry and internal light emitters.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.