Reproduce “CatPortrait: Taming Video Generative Models for Portrait Animation via In-context Conditioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziqi Ni, Zhou Fang, Shijian Wang, Rui Li, Yi Zhou
CatPortrait transfers facial motion from video or speech to another person's portrait using a rendered, texture-free 3D face sequence beside the portrait as guidance. It fine-tunes a pretrained video generator without changing its backbone or adding generator parameters. It reports the best audio-driven FVD, 113.53, and cross-reenactment AED and image quality, but not identity similarity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.