Reproduce “FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiale Xu, Shenghua Gao, Ying Shan

FreeSplatter introduces a feed-forward framework that jointly reconstructs high-fidelity 3D Gaussian representations and estimates camera poses directly from uncalibrated sparse-view images. This approach achieves state-of-the-art performance in both reconstruction quality and pose accuracy across various benchmarks, frequently surpassing methods that rely on ground truth camera poses as input.

Key insights

  • A transformer architecture can implicitly learn multi-view correspondences and geometric relationships to jointly reconstruct explicit 3D Gaussians and estimate camera poses from uncalibrated sparse views.
  • Directly predicting 3D Gaussian locations in a unified reference frame, rather than relying on intermediate depth maps or sparse point clouds, is crucial for high-fidelity pose-free reconstruction.
  • The combined two-stage training with position and pixel-alignment losses is essential for stable learning of 3D geometry from scratch in a pose-free context, overcoming the challenge of unknown initial poses.
  • Leveraging 3D Gaussian Splatting as the scene representation provides superior efficiency and quality for pose-free generalizable reconstruction compared to previous implicit or sparse representations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.