Reproduce “JVT: Just Video Transformer makes pixel-space video diffusion simple” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianjun Zhou, Hua Geng, Haoyu Guo

JVT asks whether video can be generated directly from pixels without a pretrained autoencoder. It uses a large-patch Transformer to predict clean video from noisy pixels and can jointly predict RGB and aligned disparity for camera-conditioned generation. On UCF-101, it reports FVD 136.5 versus Latte’s 477.97 at 256 × 256 on 16-frame clips. On RealEstate10K, joint RGB-disparity improves translation and AUC metrics over RGB-only, with a small increase in FVD; these camera results are specific to the reported 8-frame setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.