Reproduce “SplatMesh: Interactive 3D Segmentation and Editing Using Mesh-Based Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaichen Zhou, Lanqing Hong, Xinhai Chang, Yingji Zhong, Enze Xie, Hao Dong, Zhihao Li, Yongxin Yang and 2 more
SplatMesh integrates 3D meshes with Gaussian splatting to enable interactive 3D scene segmentation and editing, achieving superior view synthesis quality with high memory efficiency. It outperforms baselines like SuGaR, showing up to a 10% PSNR and SSIM improvement on the DTU dataset using only 9% of the Gaussians while also providing robust geometry and appearance editing capabilities.
Key insights
- Combining the explicit structure of 3D meshes with the high-fidelity rendering of 3D Gaussian Splatting offers a balanced approach for editable, photorealistic 3D scenes.
- Geometry and texture-guided mesh downsampling enables significant memory efficiency while preserving visual detail for Gaussian initialization.
- Iterative 2D-to-3D propagation, leveraging models like SAM, provides a robust and interactive method for precise 3D object segmentation from minimal user input.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.