Reproduce “ALTo: Adaptive-Length Tokenizer for Autoregressive Mask Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lingfeng Wang, Hualing Lin, Senda Chen, Tao Wang, Changxu Cheng, Yangyang Zhong, Dong Zheng, Wuyue Zhao
ALTo learns to stop mask-token generation according to the complexity of a segmentation mask, allowing an MLLM to produce shorter outputs for simpler shapes and longer ones when needed. On referring-expression segmentation benchmarks, it generally maintains or improves mask quality while reducing generated mask sequences from 32 tokens to roughly 16–19 tokens and lowering measured generation time.
Key insights
- Mask sequence length can be learned from reconstruction quality: an added token is encouraged only when its expected benefit justifies the length penalty.
- High-resolution pixel features and training on complex multi-target masks contribute to mask quality; removing the pixel encoder lowers reconstruction gIoU from 94.4 to 91.9.
- GRPO primarily adjusts efficiency rather than accuracy: on gRefCOCO validation, it reduces average length from 17.5 to 15.7 tokens with only a small change in cIoU/gIoU.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.