Reproduce “LiveSync: Iterative Self-Training for Lip-Sync Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Huaishao Luo, Huan Zhang, Guanbo Bao, Yinglu Liu, Youzheng Wu, Xiaodong He

LiveSync addresses mouth editing for new speech while preserving aligned visual context. It generates mouth-edited references from real frames, filters them, and refines the same mask-based model alongside real examples. On HDTF, it reports Sync-C 9.138 and Sync-D 6.395, best among evaluated methods. On the selected AIGC-LipSync subset, synchronization gains are not uniform.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.