Reproduce “Muse: Text-To-Image Generation via Masked Generative Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy and 4 more
Google Research's Muse introduces a masked generative transformer approach for text-to-image generation, operating in a discrete token space. This method achieves competitive image quality compared to diffusion and autoregressive models, while performing inference >10 times faster than models like Imagen and Parti, and 3 times faster than Stable Diffusion.
Key insights
- Masked modeling, similar to BERT in NLP, can be effectively adapted for image generation to achieve parallel processing and faster inference.
- Representing images as sequences of discrete tokens enables parallel prediction, differing from sequential autoregressive or iterative diffusion processes.
- Leveraging powerful, frozen pre-trained language models (T5-XXL) is essential for robust text understanding and high-fidelity image generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.