Reproduce “TTF: Temporal Token Fusion for Efficient Video-Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simin Huo, Ning LI
Temporal Token Fusion (TTF) provides a training-free and plug-and-play solution to efficiently reduce visual token counts in Video-Language Models by capitalizing on temporal redundancies. The method preserves 99.5% of baseline accuracy on Qwen3-VL-8B while reducing visual tokens by 67% and decreasing LLM inference latency by 37%.
Key insights
- Natural video sequences contain significant pointwise temporal redundancy, allowing many visual tokens from nearby frames at similar grid locations to be fused without substantial information loss.
- An efficient local window search for temporal redundancy matching, rather than computationally expensive global similarity matrix operations, is sufficient and practical for significant token reduction.
- Careful preservation and alignment of 3D positional information for compressed visual tokens are crucial for the LLM to maintain geometric consistency and avoid positional drift during decoding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.