Reproduce “iSegMan: Interactive Segment-and-Manipulate 3D Gaussians” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yian Zhao, Wanshi Xu, Ruochong Zheng, Pengchong Qiao, Chang Liu, Jie Chen
iSegMan provides an interactive framework for segmenting and manipulating 3D scenes represented by Gaussian splats. It combines epipolar geometry and visibility-based voting for precise region control and offers a versatile toolbox for edits like semantic modification, scaling, and copying, achieving real-time interaction and accurate manipulation.
Key insights
- Epipolar geometry efficiently propagates user 2D clicks across multiple views, reducing the search space for corresponding points and generating consistent 2D segmentation masks.
- Modeling 3D Gaussian selection as a visibility-based voting process from 2D segmentation masks enables robust region extraction, even in the presence of occlusions.
- Iteratively optimizing 3D Gaussian parameters based on text-guided 2D image edits allows for powerful semantic manipulation of selected 3D regions without requiring scene-specific training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.