Reproduce “Face2Face: Real-time Face Capture and Reenactment of RGB Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner
The Face2Face system from the University of Erlangen-Nuremberg, Max-Planck-Institute for Informatics, and Stanford University enables real-time facial expression transfer from a source actor to a target video using only standard RGB camera input. It successfully preserves the target's identity while accurately transferring expressions, including realistic synthesis of the mouth interior without needing 3D teeth proxies.
Key insights
- High-quality facial tracking comparable to depth-sensor systems is achievable using only standard RGB video input, enabling real-time operation.
- Separating identity and expression within a parametric face model is crucial for transferring expressions while preserving the target's unique facial features.
- Synthesizing realistic mouth interiors by retrieving and warping actual mouth regions from the target's video significantly improves visual fidelity over generic proxies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.