Reproduce “No DNN Left Behind: Improving Inference in the Cloud with Multi-Tenancy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Amit Samanta, Suhas Shrinivasan, Antoine Kaufmann, Jonathan Mace
The authors propose a shared cloud service for DNN inference, where long-lived workers host models from multiple tenants instead of provisioning a machine for each model. They argue that predictable inference costs can support fine-grained scheduling and resource sharing. Experiments report 99th-percentile latency no greater than 15% of the mean for tested workloads, but do not evaluate the proposed service.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.