Reproduce “HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maria Pilligua, Danna Xue, Javier Vazquez-Corral
HyperNVD accelerates neural video decomposition by introducing a meta-learning framework where a hypernetwork generates parameters for an Implicit Neural Representation (INR) based model. This approach enables significantly faster convergence on new videos while maintaining high reconstruction quality, making video editing workflows more efficient.
Key insights
- A pre-trained VideoMAE encoder can effectively generate compact video embeddings that capture motion and scene information, which are crucial for the hypernetwork.
- Hypernetwork-generated parameters provide a superior initialization for neural video decomposition models on new videos, accelerating fine-tuning compared to random initialization.
- The meta-learning approach allows the model to generalize across multiple videos, demonstrating scalability and maintaining strong performance even when trained on 15+ videos, despite a slight quality trade-off for broader applicability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.