Reproduce “TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiyoung Park, Hankyu Jang, Changseok Song, Wookeun Jung
TIDE adapts speculative draft models during serving by training on target-model hidden states already computed, rather than rerunning the target model to create training signals. It moves this work asynchronously to a separate training engine and controls when to train or speculate. For gpt-oss-120b on four H100s, batch 32, normalized throughput on Arabic Alpaca is 1.08, versus 0.70 with a static draft and 1.00 without speculation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.