Reproduce “Back to Pixels: Lifting Latent Video Diffusion into Pixel Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanke Song, Shuo Chen, Chris Rockwell, Guande He, Nan Liu, Desai Xie, Tianpei Gu, Yong-Hyun Park and 4 more
PixVid adapts a pretrained video diffusion transformer to generate pixels one frame at a time, replacing temporally compressed VAE latents with pixel patches. It predicts clean video and aligns controls to individual frames, avoiding block waits. On text-to-video generation, it scores 84.78 VBench versus 84.05 for Wan2.1, while control latency falls from 495 ms to 34 ms at lower throughput.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.