Reproduce “Gen Through the Lens: Camera-Controllable Video Generation via Cross-Coordinate Conditioning Fusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuai Li, Jinxu Xiang, Xinyuan Li, Hanlei Guo, Xiyang Tan, Alan Zhao, Xiaotong Zhao, Ran Zhang
Gen Through the Lens (GTL) gives image-to-video generation separate, geometry-based controls for camera motion, subject motion, and depth of field. It represents camera paths with a colored 3D grid and subjects with projected 3D boxes. In a 1.3B-model ablation, the grid lowers translation and rotation errors versus a ray map, though this does not establish performance of the 14B model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.