Reproduce “TaoMate-FVSR: Audio-Guided Facial Video Restoration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyang Li, Zeran Xu, Jingna Sun, Qijun Gan, Chenwei Zhang, Meiguang Jin, Junfeng Ma
TaoMate-FVSR restores low-quality talking-head video while correcting speech-mismatched mouth details. It mixes the encoded source with noise, then uses speech, appearance references, and recent frames to guide reconstruction. On a 150-video benchmark with synthetic degradation, it achieves the lowest perceptual distances among restoration methods (LPIPS 0.0793; DISTS 0.0858) and stronger synchronization than the baseline, while SparkVSR leads in pixel fidelity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.