Reproduce “UIKA: Fast Universal Head Avatar from Pose-Free Images” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zijian Wu, Boyao Zhou, Liangxiao Hu, Hongyu Liu, Yuan Sun, Xuan Wang, Xun Cao, Yujun Shen and 1 more

UIKA builds animatable 3D Gaussian head avatars from one or more uncalibrated, pose-free reference images. It predicts facial pixel correspondences in a shared UV map, aggregates aligned observations, and decodes them into canonical Gaussians that are driven by FLAME pose and expression controls. On multi-view NeRSemble-v2 self-reenactment, it reports PSNR 22.50 and LPIPS 0.120, outperforming GPAvatar on these metrics.

Key insights

  • Utilizing a UV-guided modeling strategy, based on pixel-wise facial correspondence, efficiently aligns multi-observation information into a unified canonical space, improving cross-image consistency.
  • The hybrid UV attention branch, integrating both screen-space and UV-space features, allows for a more comprehensive and coherent reconstruction of 3D Gaussian head avatars from diverse inputs.
  • A large-scale, identity-rich synthetic multi-view head dataset, generated using SphereHead and LivePortrait, is crucial for enhancing the model's generalization capabilities to novel identities and expressions in unconstrained scenarios.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.