Reproduce “Splatter a Video: Video Gaussian Representation for Versatile Processing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yang-Tian Sun, Yi-Hua Huang, Lin Ma, Xiaoyang Lyu, Yan-Pei Cao, Xiaojuan Qi
The University of Hong Kong's Video Gaussian Representation (VGR) converts monocular videos into explicit, dynamic 3D Gaussians, leveraging 2D foundation model priors and 3D motion regularization to achieve high-quality video reconstruction and enable versatile video processing tasks from a single input.
Key insights
- Effectively addresses the ill-posed nature of monocular 3D video reconstruction by distilling knowledge from advanced 2D foundation models as crucial regularization signals.
- The explicit, dynamic 3D Gaussian representation in camera coordinates provides a unified and manipulable framework for complex motion modeling and content manipulation, overcoming limitations of prior 2D/2.5D and implicit 3D methods.
- A hybrid polynomial and Fourier basis for Gaussian dynamics, combined with local rigidity regularization, ensures robust, flexible, and realistic 3D motion capture.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.