Reproduce “Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos
Samsung AI Cambridge researchers develop a token compression method for Large Vision-Language Models that achieves 2x higher compression rates while maintaining model accuracy through a double-forward pass strategy, enabling both efficient storage and competitive performance across generative and discriminative tasks using the LLaVA-1.5 architecture.
Key insights
- The LLM component can be leveraged to create compact, informative visual summaries
- Stage-specific adapters improve performance compared to shared weights
- Contrastive loss helps enhance both generative and discriminative capabilities
- Compressed representations encode concepts from coarse to fine-grained detail
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.