Reproduce “PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antonio Oroz, Matthias Nießner, Tobias Kirschstein
PercHead introduces a perceptual head model for single-image 3D head reconstruction, leveraging a ViT-based architecture and a novel DINOv2+SAM 2.1 perceptual loss to achieve superior photo-realism and 3D consistency, particularly in extreme-view synthesis. The framework also enables disentangled 3D editing of head geometry and style through segmentation maps and text/image prompts.
Key insights
- Leveraging foundation model features (DINOv2, SAM 2.1) for perceptual loss provides more effective and robust supervision for generative tasks, outperforming traditional losses in high-frequency detail reconstruction.
- Combining multi-view (NeRSemble, Cafca) and in-the-wild (FFHQ) datasets during training is crucial for achieving both 3D consistency and strong generalization to diverse identities and viewpoints.
- A dual-branch encoder for 2D feature extraction and an iterative 3D decoder, combined with Gaussian splatting, enables robust, high-fidelity, and view-consistent 3D head representations from single images.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.