Reproduce “DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiaxu Zhang, Xianfang Zeng, Xin Chen, Wei Zuo, Gang Yu, Guosheng Lin, Zhigang Tu
DreamDance presents a two-stage inpainting framework for 2D character animation, integrating stable 3D scene reconstruction via Gaussian Splatting with a pose-aware video inpainting model. This approach generates animations with consistent backgrounds and characters even during complex camera movements, outperforming existing methods in visual quality and user preference.
Key insights
- Reframing character animation as a two-stage inpainting problem, separating stable scene reconstruction from dynamic character animation.
- Integrating explicit 3D Gaussian fields to achieve consistent backgrounds and precise camera control in 2D animations.
- Introducing an adaptive gating strategy that controls the influence of character appearance, pose, and background information during different denoising steps of the video generation process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.