Reproduce “GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, Zexiang Xu
GS-LRM reconstructs 3D scenes from 2–4 posed images by predicting a Gaussian for each input pixel and using a transformer to combine information across views. It requires known cameras and predicts the representation in 0.23 seconds on one A100 GPU. On RealEstate10K, it reports PSNR 28.10 versus pixelSplat’s 25.89, but is limited to static scenes and surfaces within the view frustum.
Key insights
- Directly predicting explicit 3D Gaussian parameters on a per-pixel basis enables significantly faster rendering and superior detail reconstruction compared to implicit NeRF representations in LRMs.
- Integrating Plücker ray coordinates directly into input tokens provides a simple and effective method for pose conditioning in a transformer-based multi-view reconstruction system.
- A streamlined, end-to-end differentiable transformer architecture, trained on large datasets, can learn powerful generalizable priors for high-quality, sparse-view 3D reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.