Reproduce “SPEED: One-Step Pixel Diffusion for High-quality Video Frame Interpolation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zihao Zhang, Haoyu Zhao, Siqian Yang, Yidi Wu, Yudong Jiang, Zuxuan Wu

SPEED introduces a pixel-space diffusion model that achieves high-quality video frame interpolation in a single inference step. The framework demonstrates state-of-the-art perceptual quality, including an 8.8% reduction in LPIPS on the challenging SNU-FILM Extreme benchmark, while also drastically improving inference speed by up to 63.32% and significantly lowering memory consumption across various resolutions, particularly excelling at 4K.

Key insights

  • Direct pixel-space diffusion, coupled with a progressive multi-stage architecture, effectively preserves fine details and disentangles the modeling of macroscopic motion from microscopic textures.
  • An attention mechanism that specifically queries from the noisy target and aggregates keys/values from all frames allows for global context utilization while maintaining the integrity of clean input conditions.
  • By optimizing training with a drift-aware curriculum and directly predicting the clean image, high-fidelity one-step generation is achievable for diffusion models, drastically improving inference efficiency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.