Reproduce “StdGEN: Semantic-Decomposed 3D Character Generation from Single Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuze He, Yanning Zhou, Wang Zhao, Zhongkai Wu, Kaiwen Xiao, Wei Yang, Yong-Jin Liu, Xiao Han
StdGEN, developed by Tencent AI Lab and Tsinghua University researchers, generates high-quality, semantically decomposed 3D characters from a single image in less than three minutes. It reconstructs distinct body, clothing, and hair components, achieving superior geometric and textural fidelity compared to existing methods, with human evaluators preferring its overall quality in over 82% of cases.
Key insights
- Canonicalizing the input pose and generating high-resolution, view-consistent multi-view images and normal maps significantly improves 3D reconstruction quality and consistency.
- Integrating a dedicated semantic decoder and semantic-equivalent NeRF/SDF formulations within a Large Reconstruction Model allows for intrinsic 3D semantic decomposition from 2D supervision.
- Staged, iterative refinement of individual semantic layers, guided by high-resolution normals and collision losses, is crucial for achieving precise geometry and textural detail in decomposed 3D characters.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.