Reproduce “Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kepeng Xu, Li Xu, Gang He, Wenxin Yu, Yunsong Li
Researchers at Xidian University and Southwest University of Science and Technology introduce PGTFormer, an end-to-end framework for blind video face restoration that directly handles unaligned faces and diverse poses without requiring pre-alignment. The approach generates high-fidelity, temporally coherent video faces, demonstrating superior performance on various quality and temporal consistency metrics compared to prior methods while achieving faster inference.
Key insights
- Eliminating the need for explicit face pre-alignment is crucial for robust and artifact-free video face restoration, especially for low-quality and unaligned inputs.
- Semantic face parsing provides effective guidance for codebook selection in generative models, enabling robust restoration across diverse and complex face poses without fixed spatial biases.
- Integrating spatio-temporal modeling at the prior learning (TS-VQGAN) and restoration (3D Transformer in TPCP, TFR) stages is essential for achieving both high fidelity and strong temporal coherence in video face outputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.