Reproduce “Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Javad Mirzaei, Jeebak Mitra
The authors present a model for choosing how to divide large language model inference across GPUs during prompt processing and token generation. It accounts for computation, inter-GPU communication, and pipeline idle time across tensor, pipeline, and hybrid parallelism. In tests on eight A100 GPUs, preferred layouts differed by phase; these latency results are limited to tested settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.