Reproduce “3D-GIMP: When 3D Gaussian Inpainting Meets PatchMatch” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuening Tian, Dieter Schmalstieg, Shohei Mori
The 3D-GIMP method proposes a hybrid approach for view-consistent object removal in 3D Gaussian Splatting scenes, combining a single generative inpainting step on a reference view with a 3D-aware PatchMatch algorithm. This technique addresses issues like "hallucination drift" and multi-view inconsistencies, completing object removal on the IMFine dataset in 6 minutes, significantly faster than existing methods while maintaining high visual fidelity.
Key insights
- Combining a single generative prior with 3D-aware patch propagation effectively mitigates "hallucination drift" and the high computational cost of iterative multi-view diffusion.
- The use of explicit 3D Gaussian Splatting primitives, along with Poisson blending for geometrically consistent depth completion, provides a robust foundation for high-fidelity 3D inpainting.
- A visibility-aware patch-matching cost function, incorporating reprojection and occlusion resolution, enables robust cross-view propagation of inpainted textures and view-dependent effects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.