Reproduce “NeRV-Diffusion: Diffuse Implicit Neural Representations for Video Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yixuan Ren, Hanyu Wang, Hao Chen, Bo He, Abhinav Shrivastava
NeRV-Diffusion introduces an implicit latent diffusion model that synthesizes videos by directly generating implicit neural representation (INR) weights, enabling superior temporal coherence and efficient decoding while achieving competitive performance on video generation benchmarks like UCF-101 and Kinetics-600. This approach bypasses frame-wise processing, allowing for the generation of longer videos through time interpolation and smooth transitions via INR weight interpolation.
Key insights
- Generating video as a holistic implicit neural representation (INR weights) inherently ensures temporal coherence and eliminates the need for explicit temporal attention mechanisms in the generative process.
- A latent diffusion model can effectively learn to generate complex, high-dimensional INR parameters when the latent space is constrained to a well-behaved (e.g., Gaussian) distribution.
- Specialized architectural choices within the INR (e.g., channel-wise parameterization, spatiotemporal embeddings, transposed convolutions for upsampling) are crucial for achieving high-quality generative performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.