Reproduce “Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah, Godfred Manu Addo Boakye, Kwame Opuni-Boachie Obour Agyekum
A measurement study evaluated seven tactics for reducing cloud Large Language Model token usage in coding-agent applications. It demonstrated that optimal token reduction strategies are workload-dependent, with local routing and prompt compression together achieving 45–79% cloud token savings across various coding-agent workloads.
Key insights
- Optimal subsets of token-reduction tactics are highly dependent on the specific coding-agent workload, with no single "turn everything on" configuration performing universally best.
- Combining local routing (T1) and prompt compression (T2) offers a robust and effective baseline strategy, consistently yielding substantial cloud token savings across diverse workloads.
- Some tactics, like local drafting with cloud review (T4), can be counterproductive on certain workloads (e.g., short-output tasks) by increasing cloud token usage, highlighting the need for selective activation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.