Reproduce “GENSERVE: Efficient Co-Serving of Heterogeneous Diffusion Model Workloads” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fanjiang Ye, Zhangke Li, Xinrui Zhong, Ethan Ma, Russell Chen, Kaijian Wang, Jingwei Zuo, Desen Sun and 5 more
GENSERVE is a system developed by researchers from Rice University, University of Waterloo, NVIDIA, Cisco Research, and University of Washington, designed for efficient co-serving of heterogeneous text-to-image (T2I) and text-to-video (T2V) diffusion model workloads on shared GPU clusters. The system demonstrates up to a 10 percentage point increase in overall Service-Level Objective (SLO) attainment rate and a 3.1x reduction in p90 image latency compared to baselines by employing intelligent preemption and adaptive resource allocation.
Key insights
- Diffusion model inference is predictable in its per-step computational cost and naturally preemptible at denoising step boundaries, enabling fine-grained, low-overhead resource management.
- Heterogeneous T2I and T2V workloads require distinct, dynamic resource allocation strategies, where static batching or parallelism configurations are suboptimal; for example, T2I benefits from dynamic batching while T2V benefits from elastic Sequence Parallelism.
- A global, online coordination mechanism is essential to effectively balance and optimize diverse resource allocation decisions (preemption, SP, batching) across heterogeneous workloads to maximize overall SLO attainment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.