Reproduce “Preserving Source Video Realism: High-Fidelity Face Swapping for Cinematic Quality” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zekai Luo, Zongze Du, Zhouhang Zhu, Hao Zhong, Muzhi Zhu, Wen Wang, Yuling Xi, Chenchen Jing and 2 more
Researchers at Zhejiang University developed LIVINGSWAP, a video reference-guided face swapping model that delivers cinematic-quality results by directly utilizing source video details. This approach achieves high realism and temporal consistency, significantly outperforming prior methods on challenging benchmarks while reducing manual effort by approximately 40 times.
Key insights
- Directly referencing the full source video segment, rather than relying on intermediate representations or inpainting, is crucial for preserving fine-grained details, expressions, and lighting, leading to higher fidelity.
- A keyframe-based approach with temporal anchoring effectively maintains stable identity and reduces manual editing for long video sequences, providing efficient and consistent control.
- The novel 'role-reversing' training strategy allows the model to learn from and improve upon noisy, GAN-generated inputs, achieving better quality and temporal consistency than the data it was trained on.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.