Reproduce “Semantic Human Mesh Reconstruction with Textures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoyu Zhan, Jianxin Yang, Yuanqi Li, Jie Guo, Yanwen Guo, Wenping Wang
SHERT introduces a novel multi-stage pipeline for reconstructing production-ready, fully textured semantic human meshes, ensuring high-fidelity geometry, stable UV unwrapping, and animatable rigging from detailed surfaces or monocular images. The method significantly outperforms state-of-the-art approaches in non-rigid registration and monocular reconstruction metrics, while enabling text-driven texture generation.
Key insights
- Leveraging a subdivided SMPL-X model as a 'semantic guider' provides robust semantic information, stable UVs, and skinning weights crucial for industrial applications.
- Transforming 3D mesh completion into a self-supervised 2D inpainting task on the UV domain, processed in canonical space, effectively addresses missing geometric data.
- Integrating a fine-tuned Stable Diffusion model with ControlNet enables high-resolution, text-driven texture generation and inpainting on the reconstructed meshes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.