Reproduce “MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziming Zhong, Yanxu Xu, Jing Li, Jiale Xu, Zhengxin Li, Chaohui Yu, Shenghua Gao
MeshSegmenter provides a framework for zero-shot 3D semantic segmentation of untextured meshes by synthesizing realistic textures, applying 2D zero-shot segmentation models, and aggregating multi-view predictions for 3D consistency. This method achieves a 69.0% mIoU on a subset of ShapeNetPart, outperforming previous approaches.
Key insights
- Synthesizing textures for untextured 3D meshes effectively bridges the domain gap for 2D segmentation models, enhancing performance particularly for geometrically subtle regions.
- Aggregating multi-view 2D segmentation predictions with a confidence-based voting mechanism significantly improves 3D consistency and corrects errors from individual viewpoints.
- Combining both geometric (untextured) and textural (synthesized texture) information provides a more robust and accurate segmentation, leveraging complementary cues.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.