Reproduce “Bilateral Guided Radiance Field Processing” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuehao Wang, Chaoyi Wang, Bingchen Gong, Tianfan Xue

A framework uses bilateral grids to improve Neural Radiance Fields (NeRFs), disentangling camera image processing during training to eliminate artifacts and enabling consistent 3D appearance editing by propagating 2D photo edits to the entire scene. This approach achieves enhanced novel view synthesis and offers an intuitive workflow for 3D scene retouching.

Key insights

  • Differentiable bilateral grids can effectively serve as learnable proxies for complex, non-linear, local, and edge-aware camera ISP operations, both for disentanglement and consistent 3D editing.
  • Low-rank approximation of the 4D bilateral grid is crucial for generalizing sparse 2D user edits to the entire 3D scene in a coherent and view-consistent manner.
  • Appropriate regularization, specifically a sufficiently large Total Variation loss, is essential to prevent ghosting artifacts and ensure smooth propagation of transformations learned by the bilateral grids.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.