Reproduce “ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoxiao Jiang, Suyi Li, Sheng Yao, Tianyu Feng, Lingyun Yang, Dapeng Nie, Haoran Yang, Wei Wang
ServerlessT2I introduces a serverless inference system for customized Text-to-Image (T2I) workflows by deconstructing them into a DAG of model functions, implementing a GPU-resident data plane for efficient resource use, and incorporating a fair scheduling mechanism. This approach enabled up to 2x higher request rates, met 7x tighter SLOs, and reduced GPU requirements by 3x compared to existing baselines.
Key insights
- T2I inference is primarily compute-bound, resulting in substantial idle GPU memory that can be strategically utilized for caching and high-speed data transfer to alleviate bottlenecks.
- Decomposing T2I workflows into fine-grained, independently manageable model functions allows for more efficient, flexible scaling and resource management compared to monolithic deployment approaches.
- Fairness in multi-tenant serverless T2I serving requires accounting for actual GPU service time rather than simple request counts, due to the diverse resource demands of different T2I workflows.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.