Reproduce “GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generation” on OpenResearch

About this paper

Ming Qian, Zijian Wang, Minchao Sun, Jincheng Xiong, Hang Zhang, Mu Xu, Chi Wang, Baoquan Chen

GS-Voxel presents a fitting-free structured latent framework for large-scale aerial 3D Gaussian Splatting (3DGS) scene generation, deterministically converting pre-optimized, million-scale 3DGS reconstructions into a sparse voxel-aligned latent representation. The system enables image-conditioned generative modeling of expansive 3D environments, including scenes up to 1,400 m × 800 m, while maintaining high reconstruction quality with a rendered PSNR of 40.04 and SSIM of 0.98.

Key insights

• Directly converting unstructured 3DGS into a sparse, structured voxel-aligned latent is an effective and fitting-free approach for representing large-scale scenes, bypassing the need for computationally intensive refitting or global rearrangement.
• A factorized VAE that separately models geometry and attributes significantly improves compression efficiency and reconstruction quality compared to a single-stage VAE baseline.
• An optimal balance between conversion fidelity and sparse-support size for aerial scenes is achieved using a per-voxel primitive budget of K_in=16 Gaussians at a grid resolution of R=256.
• Overlap-aware tiled inference, leveraging repaint-style inpainting and feathered accumulation in boundary regions, enables seamless synthesis of 3D environments much larger than individual training crops.

Summary by alphaXiv.