Reproduce “MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunyi Li, Guo Lu, Donghui Feng, Haoning Wu, Zicheng Zhang, Xiaohong Liu, Guangtao Zhai, Weisi Lin and 1 more
Researchers from Shanghai Jiao Tong University and Nanyang Technological University developed MISC, an LMM-driven framework for ultra-low bitrate image semantic compression that simultaneously achieves high perceptual quality and consistency. The method performs effectively at bitrates down to 0.022 bpp for both natural and AI-generated images, outperforming benchmarks in combined quality metrics.
Key insights
- Integrating LMMs into compression allows for a shift to semantic-level encoding, significantly reducing bitrate while retaining critical information.
- Combining high-level semantic descriptions with low-bitrate spatial maps is essential for generative decoders to reconstruct images with both high perceptual quality and ground-truth consistency.
- Even an extremely compressed low-level image bitstream serves as a vital structural scaffold, substantially improving consistency in LMM-driven image reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.