Reproduce “FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunlong Yuan, Yuanfan Guo, Chunwei Wang, Wei Zhang, Hang Xu, Li Zhang
FreqPrior introduces a new noise initialization strategy that refines Gaussian noise in the frequency domain, addressing the variance decay issue in prior methods to enhance text-driven video generation. This approach consistently yields higher quality, more detailed, and semantically consistent videos across multiple diffusion models while reducing inference time by approximately 23% compared to FreeInit.
Key insights
- The statistical properties, specifically the Gaussian distribution, of the initial noise prior are crucial for high-quality video generation, and deviations lead to issues like "variance decay."
- Manipulating noise in the frequency domain requires careful consideration of its random variable nature, as conventional filtering can inadvertently alter its statistical distribution and degrade output.
- Efficiency in noise prior acquisition can be significantly improved by performing "partial sampling" at an intermediate diffusion timestep, reducing computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.