Reproduce “Dimba: Transformer-Mamba Diffusion Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhengcong Fei, Mingyuan Fan, Changqian Yu, Debang Li, Youqiang Zhang, Junshi Huang
Dimba is a text-to-image diffusion model developed by Kunlun Inc. that integrates Transformer and Mamba architectures to improve computational efficiency while maintaining high image quality. It achieved competitive FID scores (8.93 for Dimba-L, 8.15 for Dimba-G) on COCO with significantly reduced training resources and demonstrated superior image-text alignment on T2I-CompBench.
Key insights
- A hybrid Transformer-Mamba architecture effectively balances the global context understanding of Transformers with the linear-scaling efficiency and long-sequence handling of Mamba for text-to-image generation.
- Meticulous data curation, involving aesthetic filtering and auto-labeling for dense captions, is crucial for achieving superior image quality, artistic rendering, and robust image-text alignment in diffusion models.
- Staged training strategies, specifically quality tuning and resolution adaptation through PE Interpolation, are effective for optimizing large-scale hybrid diffusion models, accelerating convergence and improving final output quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.