Reproduce “GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiaang Duan, Shenglin Xu, Shiyou Qian, Dingyu Yang, Kangjin Wang, Chenzhi Liao, Yinghao Yu, Qin Hua and 10 more
A framework developed by researchers from Shanghai Jiao Tong University, Zhejiang University, and Alibaba Group optimizes GPU cluster scheduling for large language model workloads, integrating probabilistic demand forecasting with dynamic spot quota allocation and preemption-aware scheduling. Deployed on a 10,000+ GPU production cluster, the system reduced spot task eviction rates by up to 67.81% and increased GPU allocation rates by up to 22.79%, yielding approximately $459,715 in monthly gains.
Key insights
- Probabilistic demand forecasting, which considers organizational usage patterns and business context, provides significantly more accurate GPU resource predictions than traditional models.
- Dynamically adjusting spot instance quotas based on real-time cluster conditions (eviction rates and queuing times) is crucial for balancing high-priority task Service-Level Objectives (SLOs) with efficient low-priority resource utilization.
- Intelligent preemption strategies that quantify and minimize 'resource waste' (time since last checkpoint) for low-priority tasks lead to reduced operational costs and improved task completion rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.