Reproduce “ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anshu Raina, Yao Fu, Zhang Jiejing, Peyman Razaghi, Shiksha Patel, Ravi Gupta, Deval Shah, Adeem Jassani and 6 more
The authors introduce InferaSim, which measures complete serving-engine steps for each distinct GPU-kernel set and predicts other configurations by preserving fixed overhead while scaling divisible work. For larger deployments, it measures a reduced setup and analytically restores omitted work. Across 172 fixed-length configurations on MI355X GPUs, median TPOT error was 8.1%; evaluations were on this hardware.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.