Reproduce “GHOST 2.0: generative high-fidelity one shot transfer of heads” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alexander Groshev, Anastasiia Iashchenko, Pavel Paramonov, Denis Dimitrov, Andrey Kuznetsov
A framework named GHOST 2.0 introduces a two-stage approach for one-shot, high-fidelity head swapping, transferring a source head's identity onto a target's pose and expression. It demonstrates superior realism, identity preservation, and robustness in challenging "in-the-wild" scenarios with extreme poses and diverse hair, outperforming state-of-the-art baselines in quantitative metrics and user studies.
Key insights
- Combining separate pose and expression encoders into a single, refined motion encoder enhances disentanglement of motion from appearance, effectively preventing target identity leakage during reenactment.
- Fine-grained semantic segmentation, specifically annotating beard/facial hair as a distinct class, is crucial for accurate pixel-wise color transfer and robust blending of the entire head.
- The introduction of specialized loss functions, such as keypoint closure (`L_kpt`) and emotion loss (`L_emo`), materially improves expressiveness and fidelity of motion transfer in reenacted heads.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.