Reproduce “GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junjie Wang, Jiemin Fang, Xiaopeng Zhang, Lingxi Xie, Qi Tian

Huawei Inc. researchers developed GaussianEditor, a framework for delicate, text-guided 3D scene editing. It leverages 3D Gaussian Splatting for precise, localized modifications and achieves up to 6x faster editing (20 minutes vs. 45-120 minutes) compared to previous NeRF-based methods, while improving fidelity (FID 51 vs. 103) and user preference.

Key insights

  • Leveraging the explicit nature of 3D Gaussian Splatting enables precise, localized 3D editing, unlike implicit NeRFs.
  • Combining large language models, grounding models, and 2D diffusion models allows semantic text instructions to accurately guide 3D modifications.
  • Directly mapping 2D region of interest (RoI) to 3D Gaussians and applying masked gradient updates is crucial for delicate, non-invasive editing.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.