Reproduce “VGGTFace: Topologically Consistent Facial Geometry Reconstruction in the Wild” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xin Ming, Yuxuan Han, Tianyu Huang, Feng Xu
VGGTFace, developed by Tsinghua University researchers, reconstructs topologically consistent facial geometry from in-the-wild multi-view images by integrating a 3D foundation model with a novel topology-aware bundle adjustment. It achieved Chamfer distances of 1.18 mm on H3DS and 0.98 mm on NeRSemble, reconstructing meshes in under 10 seconds.
Key insights
- Large-scale trained 3D foundation models can provide robust initial 3D geometry and camera parameters even for challenging "in-the-wild" multi-view facial images.
- Combining raw 3D predictions from foundation models with explicit topological information (e.g., via UV maps from a 3DMM-like network) enables the creation of structured, animatable meshes.
- A Laplacian regularization term in Bundle Adjustment is effective for enforcing local geometric consistency and ensuring the completeness of a structured 3D mesh when observations are sparse or noisy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.