Reproduce “Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Aditya Gupta, Jean Sebastien Dandurand, Kai Qiu, Rohan Choudhury, Laszlo A. Jeni

SPLIT trains an adaptive image tokenizer to choose patch sizes using task loss. It begins with coarse patches and selectively splits them through a learned policy trained with PPO under a fixed token budget. On ADE20K, at about 0.7 token fraction, SPLIT allocates fine tokens to object and boundary patches more often than APT, supporting better allocation for these measured categories, not universal semantic importance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.