Reproduce “Part123: Part-aware 3D Reconstruction from a Single-view Image” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anran Liu, Cheng Lin, Yuan Liu, Xiaoxiao Long, Zhiyang Dou, Hao-Xiang Guo, Ping Luo, Wenping Wang
Part123 reconstructs 3D models from single-view images with inherent structural part information by integrating 2D segmentation models and contrastive learning into a neural rendering pipeline. This approach generates models with clear part boundaries and maintains reconstruction quality, often preferred by users over geometry-based or other 2D-to-3D lifting methods.
Key insights
- Part-awareness can be effectively integrated directly into single-view 3D reconstruction via a contrastive learning objective applied to a neural rendering framework.
- The global optimization of 3D reconstruction can implicitly resolve inconsistencies and establish cross-view correspondence among 2D semantic-agnostic segmentation masks from models like SAM.
- A graph-based method can robustly determine the number of 3D parts and initialize clustering, enabling automated, meaningful 3D segmentation from diverse 2D inputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.