Reproduce “Manga Generation via Layout-controllable Diffusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyu Chen, Dengjie Li, Zenghao Bao, Yao Zhou, Lingfeng Tan, Yujie Zhong, Zheng Zhao
Researchers at Meituan Inc. developed MangaDiffusion, a model that generates multi-panel manga pages directly from plain text stories, addressing the challenges of coherent narrative, diverse page layouts, and character consistency. The model produces clean visuals free of garbled text, making the output suitable for further editing.
Key insights
- Separating intra-panel (content generation) and inter-panel (layout and consistency) information interaction within a diffusion model effectively addresses the complexities of multi-panel manga generation.
- Utilizing a Multimodal Large Language Model (MLLM) is an effective strategy for constructing comprehensive, richly annotated datasets for specialized generative tasks where ground truth data is scarce.
- Masking out speech bubble regions during training is a practical method to produce cleaner generative outputs, improving the utility of AI-generated content by allowing for programmatic text insertion later.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.