Reproduce “TetriServe: Efficiently Serving Mixed DiT Workloads” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Runyu Lu, Shiqi He, Wenxuan Tan, Shenggui Li, Ruofan Wu, Jeff J. Ma, Ang Chen, Mosharaf Chowdhury
TetriServe, developed by researchers at the University of Michigan and Nanyang Technological University, introduces a round-based scheduler for Diffusion Transformer (DiT) models that dynamically adjusts sequence parallelism at the step level. This system improves Service Level Objective (SLO) attainment by up to 32% and offers robust performance for mixed and bursty image generation workloads.
Key insights
- Optimal step-level scheduling for DiT models with mixed workloads and varying deadlines is an NP-hard problem, necessitating efficient heuristic approaches for real-time online serving.
- DiT inference steps exhibit highly predictable runtimes, which enables the creation of an accurate cost model for informed, dynamic GPU resource allocation decisions.
- Dynamically adjusting sequence parallelism at the granularity of individual denoising steps significantly outperforms fixed parallelism by adapting to diverse request demands and meeting stringent Service Level Objectives.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.