Reproduce “Visual Geometry Foundation-Aware Gaussians for Single-Frame Surround-View Driving Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junhong Lin, Jinlong Wang, Xianda Guo, Yanlun Peng, Wei Zheng, Guoqing Liu, Hanli Wang, Tiesong Zhao and 1 more
VGGD, developed by researchers from Peking University and others, presents a feed-forward framework leveraging pretrained visual geometry foundation models to enhance 3D reconstruction from sparse surround-view cameras in autonomous driving. It achieves state-of-the-art novel-view synthesis (PSNR 24.85, SSIM 0.758, LPIPS 0.216) and improved geometric consistency (PCC 0.808) on the nuScenes benchmark.
Key insights
- Leveraging transferable upstream geometric priors from visual geometry foundation models is more effective for robust 3D reconstruction from sparse views than relying on complex decoder designs.
- Decoupling geometry-consistent features from appearance-aware features through a dual-path architecture provides stable 3D support while preserving semantic and high-frequency cues for appearance completion.
- Transient, direct supervision of metric depth during initial training effectively calibrates the depth scale, enabling stable transfer of open-domain geometric priors to specific driving camera setups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.