Reproduce “Clips2Story: Training-Free Video Storyboarding and Editing Using Multimodal Retrieval-Embedded Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junnuo Wang, Weihan Xu, Hao-Wen Dong

Clips2Story turns keyword requests into editable storyboards from shuffled clips. It describes clips, ranks them by keyword similarity, and uses an LLM to sequence footage; its ND version adds narration and matching visuals. In a 15-participant evaluation, ND scored 4.43 ± 0.14 for coherence versus REGen’s 3.17 ± 0.11, on a scale; this supports ratings in the setting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.