Reproduce “Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chendong Song, Meixuan Wang, Hang Zhou, Hong Liang, Yuan Lyu, Zixi Chen, Yuwei Fan, Zijie Zhou

Researchers from HKUST and Huawei developed an analytical framework for optimizing Attention–FFN instance provisioning in disaggregated LLM serving, which precisely accounts for stochastic workloads and cross-worker synchronization. The model accurately predicts optimal throughput and resource utilization, aligning closely with simulation results and demonstrating adaptability to varying batch sizes and workload characteristics.

Key insights

  • The stationary token load of a decode slot is an age-adjusted average, factoring in length-biasing and decode-length variance, not merely a simple arrival average.
  • Cross-worker synchronization introduces a quantifiable overhead that increases sublogarithmically with the Attention-to-FFN ratio 'r', impacting overall cycle time.
  • Optimal Attention/FFN provisioning is highly sensitive to microbatch size and workload distribution characteristics (e.g., prefill and decode lengths), necessitating an adaptive, model-driven approach.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.