Reproduce “MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Serving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiju Zhao, Junhao Hu, Rongxiao Huang, Jiaqi Zheng, Guihai Chen
Researchers from Nanjing University and Peking University developed MPIC, a system that implements position-independent multimodal context caching to accelerate MLLM inference. This system reduces Time-to-First-Token by up to 54.1% and improves throughput by 2.0x while preserving generation quality.
Key insights
- A small fraction of tokens (less than 5%) in MLLM attention matrices receive significant attention scores, indicating potential for selective processing.
- Initial tokens of an image's encoding act as "attention sinks," disproportionately attracting attention and exhibiting greater Key-Value (KV) disparity when image positions change.
- Naive full reuse of multimodal KV caches severely degrades generation quality, highlighting the need for a partial reuse strategy that balances efficiency and accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.