Reproduce “MoDM: Efficient Serving for Image Generation via Mixture-of-Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuchen Xia, Divyam Sharma, Yichao Yuan, Souvik Kundu, Nishil Talati
Researchers from the University of Michigan and Intel Labs developed MoDM, a serving system for text-to-image generation that caches final images and intelligently orchestrates a mix of diffusion models. The system substantially improves throughput by up to 3.2x and reduces tail latency while preserving image quality similar to larger models, achieving notable energy savings of up to 66.3%.
Key insights
- Caching final images, rather than intermediate features, enables cross-model compatibility and efficient storage for diffusion model serving.
- Reintroducing controlled noise to a retrieved cached image and then de-noising it with a diffusion model effectively skips initial inference steps.
- Dynamically adjusting the number of skipped steps (k) based on text-to-image similarity allows for tailored computational savings while maintaining a quality constraint.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.