Reproduce “FlashState: Unlocking Flexible Hybrid State Management for Agentic LLM Serving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenxi Li, Weili Xu, Tianyi Zhang, Jue Wang, Zhongzhu Zhou, Chenfeng Xu
The authors introduce FlashState, which captures matching attention and recurrent state at an agent request’s endpoint and selectively retains complete states under cache pressure. In a 48-session coding-agent replay with GLM-5.3-Flash, 1M-token context, and two NVIDIA B200 GPUs per engine, it achieved 4.66M prefill tokens per minute per GPU versus 1.48M for vLLM, about 3.14×, in that setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.