Reproduce “Video Diffusion Avatars: Taming Video Diffusion Models into Controllable 3D Avatars” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen, Matthias Nießner

Video Diffusion Avatars (VDAs) turn a video diffusion model into a controllable avatar from one image, separating expression, camera viewpoint, and upper-body pose. Cross-view training preserves the model’s video prior instead of building a 3D asset. On 25 BecomingLit self-reenactment cases, face-region LPIPS is 0.202 versus 0.283 for MMDM; generation takes about 12 minutes per 77-frame clip.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.