Reproduce “Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaokang Wang, Jinchang Xu, Peidong Jia, Zhijian Hao, Siyuan Qian, Fei Zhao, Rui Ma, Xiaozhu Ju and 4 more
Researchers from Peking University introduce Group-of-Latents (GoL), a generative framework for extreme-low bitrate video compression (below 0.005 bpp) that explicitly decouples spatial anchors from temporal synthesis in the latent space, leveraging Diffusion Transformers to achieve superior perceptual quality and stable spatio-temporal dynamics.
Key insights
- Explicitly separating spatial anchors from temporal generative synthesis in the latent domain allows for effective navigation of the RDP trade-off at extreme bitrates.
- Pre-trained Diffusion Transformers, when conditioned appropriately (e.g., via ControlNet), can robustly reconstruct and synthesize plausible video details from minimal transmitted information and Gaussian noise.
- Dynamic GoL training enables a single model to generalize across various GoL configurations, providing flexible bitrate adaptation at inference time without retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.