Reproduce “Architectural Implications of Agentic AI Workflows” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jirong Yang, Peizhe Liu, Chaojie Zhang, Jovan Stojkovic
Researchers at The University of Texas at Austin and Microsoft Azure characterized agentic AI workflows, revealing fragmented execution, bursty resource demands, and architectural mismatches with conventional servers. Their prototype system, Agora, implements CPU and GPU harvesting and role-aware CPU core management, achieving up to 31% increased CPU utilization, 82% higher GPU throughput, and a 2.5x reduction in GPU tail latency on commodity hardware.
Key insights
- Agentic AI workloads are fundamentally fragmented, involving dynamic graphs of interleaved LLM inferences and host-side tool invocations, with host CPU frequently on the critical path and repeatedly crossing the CPU-GPU boundary.
- These workflows exhibit heterogeneous and bursty resource demands, leading to significant resource stranding (e.g., average CPU 6-31%, GPU SM activity below 55%) despite transient near-saturation bursts on conventional servers.
- Current server architectures suffer from inefficient homogeneous CPU provisioning and degraded microarchitectural locality (low IPC, high cache misses, branch mispredictions) when multiplexing agent tasks, leading to increased coordination overhead and pipeline stalls.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.