Reproduce “MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Youngrae Kim, Qixin Hu, C.-C. Jay Kuo, Peter A. Beerel
A training-free framework named MemRoPE enables unbounded video generation, maintaining subject identity and visual fidelity over minute-to-hour durations by utilizing dual Exponential Moving Average (EMA) memory tokens and online Rotary Position Embedding (RoPE) indexing within a fixed-size KV cache. This approach achieves superior VBench-Long scores and strong user preference compared to existing baselines, extending the capabilities of pretrained autoregressive video diffusion models.
Key insights
- Long-horizon coherence in video generation benefits more from continuously evolving and aggregating past context (
- remembering better
- ) rather than simply retaining more raw frames.
- Decoupling content (unrotated keys) from positional information (RoPE) in the KV cache is essential for enabling correct and valid temporal aggregation of context.
- A dual-stream EMA memory effectively manages both global identity (long-term stream) and recent dynamics (short-term stream), crucial for preventing stagnation and abrupt visual shifts over extended durations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.