Reproduce “SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Quanjian Song, Donghao Zhou, Jingyu Lin, Fei Shen, Jiaze Wang, Xiaowei Hu, Cunjian Chen, Pheng-Ann Heng
SceneDecorator introduces a training-free framework for scene-oriented story generation, addressing scene planning and long-term scene consistency. It leverages VLM-guided decomposition and masked attention mechanisms to produce visually coherent narratives, demonstrating superior scene alignment and consistency in both automatic metrics and user studies.
Key insights
- Leveraging Vision-Language Models for a global-to-local scene planning process is effective for ensuring scene-level narrative coherence across sequential images.
- Maintaining scene consistency across diverse story narratives requires selectively sharing background visual information while allowing foreground subjects to retain their unique attributes.
- An extrapolable noise blending scheme can achieve long-term scene consistency across an arbitrary number of stories without a linear increase in memory usage.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.