Reproduce “Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yipeng Liu, Yingqiang Zhang, Feifei Li, Huanchen Zhang
Live progress reporting gives LLM serving systems a more reliable basis for KV-cache decisions during tool calls than pre-execution duration prediction. Across coding-agent traces, the approach exposed strong or weak progress signals for roughly 59–72% of tool time, reduced p90 post-tool time to first token by about 21% overall and up to 35% for calls lasting at least 10 seconds, while adding less than 1% runtime overhead.
Key insights
- Execution-state signals outperform historical duration estimates because they adapt to runtime conditions and provide increasingly accurate remaining-time estimates as a tool progresses.
- At the midpoint of calls, reported progress achieved about 21.8% median relative remaining-time error versus 32.6% for the strongest history-based baseline; near completion, its error stayed below 10% while the baseline was around 80%.
- Coverage is the main constraint: progress can be recovered for many local and instrumentable tools, but opaque remote APIs, human waits, and silent downloads may provide no usable intermediate state.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.