Reproduce “500xCompressor: Generalized Prompt Compression for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zongqian Li, Yixuan Su, Nigel Collier
Researchers at the University of Cambridge developed the 500xCompressor, a method that compresses LLM text prompts by up to 480 times using Key-Value values from special tokens. It achieved an 82% Rouge-L F1 for text regeneration with 16 compressed tokens and retained 62-73% of full-context performance on extractive question answering, consistently outperforming previous compression baselines.
Key insights
- Using Key-Value (KV) values from compressed tokens consistently outperforms embedding-based methods for information preservation, particularly at high compression ratios.
- While performance improves with more compressed tokens, the gains exhibit diminishing returns, indicating that not all additional tokens contribute equally to information retention.
- There is a roughly logarithmic trade-off between compression ratio and performance, with performance degradation becoming more pronounced at extreme compression levels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.