Reproduce “System X: An LLM Decision Model That Replies from Its Own KV Cache” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Syed Usama Khalid Bukhari, Syed Safwan Khalid, Muhammad Aamir
System X adds a reply option to action selection and generates text from the prompt’s existing key–value cache, avoiding a second prefill. In a matched Apple M4 Pro test, it cut latency 37% at 4,161 tokens, with both paths generating 32 reply tokens; this establishes a time saving, not fewer generated tokens or better reply quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.