Reproduce “Multi-Image Visual Token Pruning in Large Visual Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rongyang Zhang, Chengqiang Lu, Cong Li, Hongchao Gu, Tingjia Shen, Xuyang Zhi, Qimeng Wang, Yan Gao and 4 more
Adaptive Visual Token Pruning (AVTP) is a training-free framework from the University of Science and Technology of China and Xiaohongshu Inc. designed to improve the efficiency of Large Vision-Language Models (LVLMs) in multi-image scenarios by dynamically pruning visual tokens. The method enables full compatibility with efficient attention mechanisms like FlashAttention2 and achieved a 2x inference speedup for Qwen3VL-8B while retaining 96.1% of its original accuracy.
Key insights
- Visual token pruning not only reduces inference latency but can also improve LVLM performance by shortening input sequences, thereby mitigating performance degradation from excessively long context lengths.
- Compatibility with attention acceleration mechanisms like FlashAttention is crucial; incompatible pruning methods introduce significant memory overhead, negating efficiency gains and leading to Out-Of-Memory (OOM) issues.
- Adaptive, architecture-aware pruning, which dynamically selects pruning layers and adjusts pruning rates per image based on relevance, consistently achieves better accuracy-latency trade-offs across diverse LVLM architectures.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.