Reproduce “SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov and 1 more
SnapFusion enables text-to-image diffusion models to generate 512x512 images on mobile devices like the iPhone 14 Pro in under two seconds, achieving competitive image quality and often surpassing the fidelity of larger, slower models with significantly fewer inference steps.
Key insights
- Robust training allows for stable architectural changes within the UNet, such as removing cross-attention or ResNet blocks, without significantly degrading image quality, which is critical for efficiency-driven model evolution.
- Direct distillation for reducing denoising steps (e.g., 16 steps to 8 steps) yields superior FID and CLIP scores compared to progressive distillation for the same target step count.
- Integrating Classifier-Free Guidance (CFG) directly into the step distillation loss (CFG-aware distillation) is essential for effectively preserving and improving CLIP scores when significantly reducing inference steps, particularly at higher guidance scales.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.