Reproduce “4Director: Controlling Video World Models with Rigid 3D Geometry” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei Cao, Hao Zhang, Vikram Voleti, Yuqun Wu, Mallikarjun B R, Shimon Vainer, Mark Boss, Yaoyao Liu
4Director turns images into controllable videos by reconstructing selected objects as complete 3D meshes and keyframing each object’s rigid motion alongside the camera. It renders the scene as depth video to guide a pretrained video generator. On 100 held-out clips, it achieves 60.4 Identity-Gated IoU versus VerseCrafter’s 54.8, with nearly identical recognition rates; this supports better placement, not identity preservation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.