Reproduce “zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Saibo Geng, Andrea Gentilini, Xinxian Ma, Nathan Ranchin, Yunzhen Yao, Robert West

zip2zip++ adapts an LZW-based dynamic vocabulary for pretrained language models to reduce quality loss from token compression. It protects digits, adjusts token positions, and changes embeddings and decoding to handle imperfect compressed predictions. On Phi-3.5-mini, two-shot GSM8K accuracy is 0.68, above zip2zip’s 0.36 but below the uncompressed control’s 0.75; compression reduces token counts, but faster inference is unmeasured.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.