Reproduce “Generative Image Dynamics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhengqi Li, Richard Tucker, Noah Snavely, Aleksander Holynski
Google Research developed a system to infer and generate realistic, continuous oscillatory motion from a single static image. It achieves this by modeling motion as spectral volumes in the frequency domain, learned through a Latent Diffusion Model, enabling the creation of high-quality, temporally coherent animations.
Key insights
- Spectral volumes provide a compact and effective representation for learning and generating natural, oscillatory motion with long-term temporal coherence.
- Specialized LDM training techniques, such as frequency adaptive normalization and coordinated denoising, are essential for learning complex, structured data like spectral volumes.
- Combining classical computer graphics representations with modern diffusion models allows for generating realistic and diverse motion conditioned on a single image.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.