Reproduce “3D Scene Prompting for Scene-Consistent Camera-Controllable Video Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

JoungBin Lee, Jaewoo Jung, Jisang Han, Takuya Narihira, Kazumi Fukuda, Junyoung Seo, Sunghwan Hong, Yuki Mitsufuji and 1 more

3DScenePrompt continues camera-controlled video by combining recent frames for evolving motion with static 3D spatial prompts rendered from requested viewpoints. It builds the map by removing dynamic points, then conditions CogVideoX on both context types. On DynPose-100K revisited views, it improves PSNR to 13.0468 versus 12.1471 for DFoT, while MEt3R is 0.124189 versus 0.183202; revisited views use static-region scores, and output is chunked.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.