Reproduce “TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jihao Zhao, Shuaishuai Zu, Pengfei Li, Biao Qin, Guanghuan Fang, Ben Feng, Baochuan Yang
TraceSpec evaluates speculative decoding across agent trajectories and trains draft models by supervising intermediate assistant turns, using disagreement conditioned on context, phase, workload, and tool history to allocate a fixed prediction budget. In matched late BrowseComp-Plus actions, DSpark accepted length fell from 6.194 to 5.322 tokens per verification round. Replay throughput improved 12.5%–22.6%, without showing better task correctness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.