Reproduce “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz and 9 more

A team at Stability AI introduces Rectified Flow Transformers as an alternative to diffusion models for high-resolution image synthesis, incorporating novel noise sampling techniques and a Multimodal Diffusion Transformer architecture. This approach matches and exceeds the performance of state-of-the-art diffusion models, generating high-quality images in as few as 4-8 sampling steps.

Key insights

  • Rectified Flow models, when equipped with optimized noise sampling and a suitable architecture, can surpass the performance of state-of-the-art diffusion models for high-resolution image synthesis.
  • A Multimodal Diffusion Transformer (MM-DiT) architecture enabling bi-directional information flow between image and text tokens significantly improves the model's text understanding and image generation quality, particularly for complex prompts and typography.
  • The performance of Rectified Flow models scales predictably with increases in model size, dataset size, and training compute, suggesting a clear path for further improvements through scaling.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.