Reproduce “Messages, Not Tokens: Grounded Coresets for Faithful VLM Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Long Qian, Jiaqi Wei, Bingke Zhu, Yingying Chen, Jinqiao Wang
Grounded Message Coreset Pruning (GMC) introduces a training-free framework that addresses visual compression for Vision-Language Models by preserving collective, query-conditioned messages rather than individual tokens. This approach helps models retain high capability, achieving up to 100.36% of full model performance even at 90% visual token reduction, while also providing a 1.258x end-to-end speedup and reducing prompt Key-Value cache size by 73.94%.
Key insights
- Faithful VLM compression requires preserving the collective, signed 'grounded messages' consumed by the language decoder, rather than focusing solely on individual token saliency.
- Decomposing compression into complementary support allocation and message realization allows for a principled approach to jointly determine where evidence is retained and what information it carries.
- Theoretical bounds connect low-level compression errors in support allocation and message realization to higher-level metrics such as signed-message distortion, visual innovation, and candidate-margin stability, providing auditable fidelity measures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.