Reproduce “SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zijie Wu, Chaohui Yu, Yanqin Jiang, Chenjie Cao, Fan Wang, Xiang Bai
SC4D introduces an efficient, sparse-controlled framework for generating dynamic 3D objects (4D content) from single-view videos. It consistently produces visually coherent and temporally stable 4D assets with high motion fidelity, outperforming previous methods and enabling motion transfer.
Key insights
- Decoupling motion (sparse control points) and appearance (dense Gaussians) simplifies the challenging task of learning complex 4D dynamics from single-view videos, leading to more stable and controllable results.
- Informed initialization of dense Gaussians and an explicit alignment loss are crucial for maintaining object fidelity and preventing degradation during optimization with 2D diffusion priors.
- Capturing motion within a sparse set of control points enables effective disentangled motion transfer, allowing learned movements to be applied to diverse, text-guided 4D entities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.