Reproduce “FlexComp: One Model for Every Ratio in Context Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaiyan Zhao, Zhongtao Miao, Akiko Aizawa, Yoshimasa Tsuruoka
FlexComp trains a single soft context compressor to operate at multiple memory budgets, replacing separate fixed-ratio models and enabling input-dependent allocation. Across MRQA tasks, adaptive cascade and predictor-based routing achieve roughly 158–266× average compression, with under 0.7 F1 loss in reported settings; the predictor also cuts context KV-cache memory by 50.5% and raises throughput by 47% at an 8,192-token context.
Key insights
- Joint multi-budget training generally matches independently trained specialists, with differences usually around one F1 point and occasional gains at highly compressed settings.
- Required memory is strongly input-dependent: some examples are solved with one memory token, whereas roughly 11%–26% of solvable examples require the largest tested budget.
- More memory is not always better; larger representations can preserve distracting or irrelevant details, allowing highly compressed versions to outperform full-budget inputs on about 11.8%–12.8% of in-domain examples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.