Reproduce “AdaHuman: Animatable Detailed 3D Human Generation with Compositional Multiview Diffusion” on OpenResearch

About this paper

Yangyi Huang, Ye Yuan, Xueting Li, Jan Kautz, Umar Iqbal

NVIDIA researchers develop AdaHuman, a framework that generates highly detailed and animation-ready 3D human avatars from single in-the-wild images through pose-conditioned joint diffusion and compositional 3D Gaussian Splatting refinement, achieving state-of-the-art performance with 21.46 PSNR on CustomHumans and 24.64 PSNR on novel pose synthesis while enabling A-pose avatar generation from arbitrary input poses to solve self-occlusion issues that plague existing methods, with 88.3% user preference over competing approaches like SiTH and the ability to synthesize realistic cloth deformations and fine-grained details through visibility-aware composition of local body part refinements at higher resolution, directly addressing critical animatability challenges in 3D content creation pipelines.

Key insights

• Interleaving 3DGS reconstruction within the diffusion process rather than applying it only at the end is crucial for maintaining 3D consistency across views
• Pose conditioning with semantic maps and camera ray embeddings enables generation of standard A-pose avatars from arbitrary input poses without requiring standard-pose training data
• Visibility-aware 3DGS composition based on view coverage and alpha gradient magnitude effectively prevents floating artifacts when merging local refinements
• Direct avatar reposing captures realistic cloth deformation better than SMPL-based deformation for non-rigid clothing

Summary by alphaXiv.