Reproduce “PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiqi Zhang, Fangzheng Jiao, Tian Tang, Boyu Tian, Hangyu Wang, Qiaoling Chen, Guoteng Wang, Zhen Jiang and 8 more
PlexRL, a system developed by researchers at National University of Singapore, Beihang University, and Shanghai Qiji Zhifeng Co., Ltd., introduces cluster-level orchestration for large language model (LLM) execution in reinforcement learning with verifiable rewards (RLVR) workflows. It decouples algorithmic control from model execution to multiplex LLM services across multiple jobs, achieving up to 37.58% reduction in GPU-hour cost and increasing workload capacity by 1.8 times.
Key insights
- Idle periods within individual RLVR jobs are largely anti-correlated across different jobs, presenting a significant opportunity for resource reclamation through cluster-level multiplexing.
- Decoupling algorithmic control from model execution is crucial for enabling global resource orchestration by the cluster scheduler and for promoting greater flexibility in RLVR algorithm development.
- Affinity-aware scheduling and robust, hierarchical state management are essential to enable efficient time-slicing and multiplexing of large LLMs across shared GPU resources, minimizing expensive context switches and model migrations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.