Reproduce “Temporal Context Mining for Learned Video Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xihua Sheng, Jiahao Li, Bin Li, Li Li, Dong Liu, Yan Lu
Temporal Context Mining (TCM) for learned video compression extracts richer, multi-scale contexts from propagated features, rather than decoded frames, to enhance compression efficiency. The framework achieves a 14.4% bit rate saving over H.265-HM (PSNR) and 21.1% over H.266-VTM (MS-SSIM) on HEVC Class RGB, while reducing 1080p frame decoding time by 98.7% compared to DCVC by avoiding auto-regressive models.
Key insights
- Learning temporal contexts from propagated internal features preserves significantly more information than using only decoded frames, leading to superior compression performance.
- Multi-scale temporal contexts, when strategically integrated (re-filled) across multiple stages of the codec, effectively capture and utilize the complex, non-uniform spatio-temporal dynamics in videos.
- It is possible for learned video codecs to achieve competitive or superior compression efficiency against highly optimized traditional standards like H.266/VVC, particularly in perceptual metrics like MS-SSIM, while also providing practical decoding speeds by avoiding auto-regressive models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.