Reproduce “The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mukund Athmaram, Manoj Rao, Liang Ma
The authors develop a calibrated latency model and precision allocator for LLM serving. The model estimates whether computation, weight loading, or KV-cache access limits execution, rather than treating weight bytes as a universal proxy for speed. In one MI355X MoE decode test, it predicts latency with 5.4% MAPE, versus 17.5% for bytes; this result is specific to that workload.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.