Reproduce “Fast Recovery for LLM Serving via Decoupled Device Memory Lifetime in Dynamo” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Schwinn Saereesitthipitak, Mohammed Abdulwahhab, Hannah Zhang, Dan Feigin, Neelay Shah, Maksim Khadkevich, Itay Neeman, Vikram Sharma Mailthody and 1 more

Dynamo restores serving capacity after an engine failure by promoting a ready Shadow Engine that shares surviving model weights through GPU Memory Service and installs fresh KV backing; interrupted requests are replayed. In tests on eight-GPU DGX B200 replicas, vLLM on GLM-5.2 took 144.4 seconds to warm-restart, versus 5.6 seconds with shadow recovery, a 25.8× speedup. This fast path applies only when GPUs and resident allocations remain healthy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.