Reproduce “SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yilai Liu, Xin Zhang, Shiyuan Zhang, Hongyang Du
SlotMem introduces a framework for narrative long video generation that maintains consistent recurring character identities by employing a character-addressable internal memory system. This system disentangles character features into dynamically updated memory slots, resulting in higher scores for character consistency metrics like VBench Human Anatomy (0.9480) and ViStoryBench Character Similarity (0.8603) while preserving overall video quality.
Key insights
- Disentangling character identity from incidental visual factors into compact, role-wise memory slots significantly improves consistency compared to holistic frame-level memory approaches.
- Dynamic and continuous updating of character memory based on new observations is crucial for maintaining identity over long narratives, overcoming the limitations of static memory systems.
- Localizing character-specific regions for both memory extraction and targeted injection via character-wise cross-attention effectively mitigates interference from other characters or background elements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.