Reproduce “DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dongzhi Jiang, Renrui Zhang, Haodong Li, Zhuofan Zong, Ziyu Guo, Jun He, Claire Guo, Junyan Ye and 4 more
DraCo introduces an interleaved visual and textual reasoning paradigm for unified Multimodal Large Language Models (MLLMs), generating a low-resolution visual draft for self-correction. This approach significantly enhances text-to-image generation quality, achieving an 8% improvement on GenEval and a 0.91 point increase on ImagineBench for challenging and rare concept prompts.
Key insights
- Interleaving visual drafts with textual verification enables MLLMs to perform concrete, iterative visual planning and self-correction, which is superior to abstract text-only reasoning for visual tasks.
- Excluding VAE features during draft verification allows for high-level semantic understanding and substantial corrections without being constrained by low-level draft imperfections.
- Tailored guidance strategies, such as DraCo-CFG, are critical for effectively managing multiple multimodal conditioning signals to achieve optimal results in complex generative processes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.