Reproduce “Beyond Microservices: Testing Web-Scale RCA Methods on GPU-Driven LLM Workloads” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dominik Scheinert, Alexander Acker, Thorsten Wittkopp, Soeren Becker, Hamza Yous, Karnakar Reddy, Ibrahim Farhat, Hakim Hacid and 1 more
Researchers from logsight.ai, Technische Universität Berlin, and Technology Innovation Institute systematically evaluated 24 automated Root Cause Analysis (RCA) methods on a best-practice LLM inference stack under controlled fault injections. The study reveals that while modern metric-based RCA methods show promise, existing trace-based and multi-source methods often misattribute faults or struggle with GPU-related anomalies, highlighting specific observability gaps in LLM-serving environments.
Key insights
- Existing RCA methods frequently misattribute faults in LLM stacks, often pointing to gateway or head components rather than specific faulty worker Pods, suggesting a bias towards symptoms rather than true origins.
- GPU-centric anomalies (e.g., throttling) are largely undetected by current RCA methods because standard GPU telemetry (utilization, memory) often remains unchanged during such events, revealing a critical gap in observability for accelerator-bound workloads.
- LLM-specific characteristics like dynamic request batching and fragmented execution paths render traditional trace-based RCA methods largely ineffective compared to their performance in traditional microservice environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.