Reproduce “Compressing Human Body Video with Interactive Semantics: A Generative Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bolin Chen, Shanzhi Yin, Hanwei Zhu, Lingyu Zhu, Zihan Zhang, Jie Chen, Ru-Ling Liao, Shiqi Wang and 1 more
The Interactive Human Video Coding (IHVC) framework compresses human body video at ultra-low bitrates by encoding signals into compact, interactive semantic embeddings. This approach achieves competitive Rate-Distortion performance and enables direct manipulation of elements like head pose and body location from the compressed bitstream, crucial for metaverse applications.
Key insights
- Disentangling complex human motion into configurable and compact semantic embeddings (e.g., head pose, body pose, location) enables significant bitrate reduction and direct interactivity within the compressed bitstream.
- Optimizing the number of transmitted semantic parameters by deriving constant or implicitly linked components (e.g., body shape, most body joints from key-reference frame) enhances compression efficiency.
- Combining a traditional video codec (VVC) for high-quality keyframe texture with deep generative models (GANs) for inter-frame synthesis, guided by explicit semantic parameters, yields superior performance at ultra-low bitrates and offers intrinsic semantic control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.