Reproduce “HexAGenT: Efficient Agentic LLM Serving via Workflow- and Heterogeneity-Aware Scheduling” on OpenResearch

About this paper

You Peng, Youhe Jiang, Wenshuang Li, Xu Xu, Ke Zhou, Jiawei Jiang, Chen Wang, Binhang Yuan

HEXAGENT is a scheduling framework designed for efficiently serving multi-step agentic Large Language Model (LLM) workflows in prefill-decode disaggregated and heterogeneous GPU clusters. It introduces a workflow- and heterogeneity-aware scheduler that prioritizes end-to-end workflow completion, achieving up to a 24.5% reduction in average Req99 latency on heterogeneous clusters compared to baseline methods.

Key insights

• Optimizing for end-to-end workflow completion, rather than individual LLM call latency, is crucial for improving user experience in agentic applications, leading to substantial performance gains.
• Jointly considering workflow structure and hardware heterogeneity during scheduling significantly enhances resource utilization and reduces tail latencies in P-D disaggregated LLM serving clusters.
• The workflow-level prioritization signal in HEXAGENT provides robustness against moderate inaccuracies in estimated call service times, making the system practical in dynamic environments.

Summary by alphaXiv.