Reproduce “Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting” on OpenResearch

About this paper

Xiangyu Sun, Runnan Chen, Mingming Gong, Dong Xu, Tongliang Liu

Intern-GS, developed by researchers from the University of Sydney, University of Melbourne, and University of Hong Kong, proposes a framework for sparse-view 3D Gaussian Splatting that utilizes vision foundation models for dense initialization and multi-view appearance/depth refinement. This approach achieves state-of-the-art rendering quality on LLFF (20.49 PSNR), DTU (20.34 PSNR), and Tanks and Temples (22.67 PSNR) datasets by providing geometrically consistent and perceptually realistic 3D reconstructions from limited input views.

Key insights

• Leveraging rich prior knowledge embedded within vision foundation models can effectively bridge the information gaps inherent in sparse-view 3D reconstruction.
• A robust and dense initialization, specifically a multi-view stereo guided method, significantly outperforms traditional SfM-based approaches for explicit 3D representations like 3DGS under sparse and low-texture conditions.
• A hybrid regularization mechanism, combining geometric constraints from depth priors and appearance refinement from diffusion models, is critical for achieving both structural accuracy and perceptual realism in novel viewpoints.

Summary by alphaXiv.