Reproduce “Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vikash Sehwag, Xianghao Kong, Jingtao Li, Michael Spranger, Lingjuan Lyu
Sony AI researchers demonstrate methods to train competitive text-to-image diffusion models for under $2,000 and within three days on a single 8xH100 GPU machine. This work introduces a "deferred masking" strategy and a comprehensive pipeline that drastically reduces computational resources, making T2I model development more accessible.
Key insights
- Processing all image patches with a lightweight module before applying high masking ratios preserves semantic information, enabling significant computational savings.
- Combining publicly available real and synthetic images for training can yield competitive text-to-image model performance, reducing the reliance on massive, exclusive datasets.
- Training large-scale text-to-image diffusion models can be made accessible to researchers with limited computational budgets by optimizing architectural and training strategies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.