Reproduce “TempoMaster: Efficient Long Video Generation via Next-Frame-Rate Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yukuo Ma, Cong Liu, Junke Wang, Junqi Liu, Haibin Huang, Zuxuan Wu, Chi Zhang, Xuelong Li
TempoMaster presents an efficient framework for long video generation, employing a novel next-frame-rate prediction paradigm to mitigate temporal inconsistencies and high computational costs. The model achieved state-of-the-art Vbench scores of 80.30 for 500-frame videos and 80.76 for 121-frame videos, while demonstrating superior perceptual quality in human evaluations and generating coherent videos over 1500 frames.
Key insights
- Decoupling high-level temporal semantics from low-level visual details allows for maintaining global consistency with sparse frames while efficiently refining local details.
- Framing long video generation as predicting successive frame rates (rather than individual frames) enables a hierarchical approach that avoids error accumulation characteristic of traditional autoregressive models.
- Parallel inference, leveraging the multiway tree structure of the hierarchical generation, dramatically reduces computational complexity from quadratic to an exponentially faster rate (e.g., O(N^2 / 4^K)), making minute-long video generation feasible.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.