Reproduce “Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen Ju, Haicheng Wang, Haozhe Cheng, Xu Chen, Zhonghua Zhai, Weilin Huang, Jinsong Lan, Shuai Xiao and 1 more
The Turbo plug-in accelerates Vision-Language Large Models (VLMs) by implementing an informativity-driven data de-redundancy strategy, which identifies and prunes superfluous input tokens across various understanding and generation tasks. This approach achieves up to 2.8x throughput acceleration with minimal performance degradation and operates as a universal, plug-and-play module.
Key insights
- Significant data redundancy exists within VLM input sequences, which can be strategically reduced without substantial information loss by considering both token similarity and semantic contribution.
- Quantifying token "informativity" by combining mutual redundancy and semantic value provides a robust criterion for selective token pruning, balancing efficiency and performance preservation.
- A data-centric acceleration approach can serve as an orthogonal and complementary method to existing model-centric optimizations, offering broad compatibility across diverse VLM architectures and tasks without re-training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.