Reproduce “Surprise Forcing: What to Remember, When to Skip in Long Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuwei Shi, Zhen Li, Muyao Niu, Chuanhao Li, Bo Zheng, Kaipeng Zhang, Yinqiang Zheng
Surprise Forcing is a training-free inference framework that enhances long-duration video generation by dynamically managing historical context and computational resources based on "surprise" signals derived from a model's intermediate representations. The approach improves subject consistency and visual quality in minute-scale videos while maintaining real-time throughput by selectively retaining important frames and adaptively allocating denoising steps.
Key insights
- Online "surprise" signals derived from a generative model's intermediate representations provide an inexpensive and effective way to dynamically manage both historical context and computational allocation during long video inference.
- Intelligently selecting which historical frames to retain in an external memory, based on their 'surprise' (novelty and deviation from existing memory), significantly improves long-range visual consistency in generated videos.
- Adaptively skipping denoising steps for "easy" video chunks, identified by their low intra-chunk temporal variation, allows for computational savings and increased throughput without substantial quality degradation in complex segments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.