Reproduce “HFX: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scaling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zahra Yousefijamarani, Xinglu Wang, Qian Wang, Morgan Lindsay Heisler, Taha Shabani, Niloofar Gholipour, Parham Yassini, Hong Chang and 7 more

HFX coordinates scheduling and elastic scaling across LLM replicas for requests with different latency requirements. Its dispatcher predicts when an instance can safely accept work, admits urgent requests within a token budget, and adds capacity when safe placement is unavailable. On the four-task benchmark, HFX-Scaling achieves up to 4.44× SLO attainment over Round-Robin and 2.59× over SCORPIO, within tested workloads and Ascend hardware.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.