Reproduce “HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments” on OpenResearch

About this paper

Ji Liu, Zhihua Wu, Dianhai Yu, Yanjun Ma, Danlei Feng, Minxu Zhang, Xinxuan Wu, Xuefeng Yao and 1 more

HeterPS is a distributed deep learning framework from Baidu Inc. and USTC that optimizes the training of large-scale Deep Neural Networks across heterogeneous computing resources. It utilizes a Reinforcement Learning-based layer scheduler and dynamic resource provisioning to minimize monetary cost while satisfying throughput constraints, demonstrating up to 14.5 times higher throughput and hundreds to thousands of percent lower costs compared to existing methods.

Key insights

• Fine-grained, layer-level scheduling by an RL agent can significantly improve cost efficiency and resource utilization in heterogeneous distributed deep learning environments.
• Dynamically provisioning computing resources based on a detailed cost model and workload characteristics, rather than static heuristics, leads to substantial monetary savings while meeting performance targets.
• A holistic system design that integrates intelligent scheduling, dynamic resource provisioning, and optimized data management (e.g., caching, hotness management, communication overlap) is crucial for maximizing throughput and cost-effectiveness in distributed heterogeneous setups.

Summary by alphaXiv.