Reproduce “LayerDiff: Exploring Text-guided Multi-layered Composable Image Synthesis via Layer-Collaborative Diffusion Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Runhui Huang, Kaixin Cai, Jianhua Han, Xiaodan Liang, Renjing Pei, Guansong Lu, Songcen Xu, Wei Zhang and 1 more

LayerDiff introduces a text-guided, layer-collaborative diffusion model that generates multi-layered, composable images, enabling fine-grained control and manipulation of individual elements. The model, a collaboration between Sun Yat-sen University and Huawei Noah’s Ark Lab, achieves competitive image quality for two-layered compositions and supports applications like layer inpainting and style transfer.

Key insights

  • A layer-collaborative attention block within the diffusion model effectively manages inter-layer relationships for global coherence and intra-layer content generation via layer-specific prompts.
  • A layer-specific prompt enhancer refines individual layer prompts by leveraging global context, improving textual control and inter-object relationships.
  • The Self-Mask Guidance (SMG) sampling strategy uses predicted masks to enhance content localization and fidelity within each layer during inference.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.