Reproduce “PERF: Panoramic Neural Radiance Field from a Single Panorama” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guangcong Wang, Peng Wang, Zhaoxi Chen, Wenping Wang, Chen Change Loy, Ziwei Liu
PERF presents a framework for constructing a panoramic Neural Radiance Field from a single 360-degree image, enabling 3D roaming and novel view synthesis in complex indoor scenes. The method achieves superior rendering quality and semantic coherence in occluded regions, demonstrated by an 81% user preference on real-world datasets and improved PSNR, SSIM, and LPIPS metrics on the Replica dataset.
Key insights
- Large-scale pre-trained models (like Stable Diffusion for RGB and monocular depth estimators) can be effectively integrated into a NeRF framework to infer and generate content for extensive occluded regions in panoramic scenes.
- Maintaining geometric consistency in single-view 3D scene generation is crucial; the progressive inpainting-and-erasing strategy successfully prevents newly generated geometry from conflicting with previously visible scene parts.
- Projecting panoramic views into normal field-of-view images allows leveraging existing single-view 2D vision models for monocular depth estimation and inpainting within a 360-degree 3D reconstruction pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.