Reproduce “End-to-End Context Compression at Scale” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ang Li, Sean McLeish, Haozhe Chen, Nimit Kalra, Zaiqian Chen, Artem Gazizov, Venkata Anoop Suhas Kumar Morisetty, Bhavya Kailkhura and 7 more

LCLMs compress prompts by encoding input groups into learned latent tokens that the decoder reads instead of the source context. This shortens prefill, though accuracy can decline at higher compression ratios. On RULER at 4K, a 4× LCLM achieved 15.9× faster time to first token than the fastest KV-cache method within one accuracy point, on one H200.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.