Reproduce “Differentiable Virtual-Slot Partitioning for Adaptive Tokenization in Vision Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Gaoxiang He, Bin Zhu, Bo Xie, Xiaotong Yu, Yi Chen, Hongfei Yan

DVSP reduces vision-transformer work by learning where to use coarse or fine image patches while preserving a regular grid. During training, four virtual slots blend coarse and fine tokens with attention mass m(α) = (1 + 3α)/4; exact endpoint matching requires aligned windows, token-wise layers, and specified readouts. On COCO val2017 detection, it uses 3,303 rather than 9,216 tokens and scores 57.72 Box AP, 97.9% of dense performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.