Reproduce “LAION-SG: An Enhanced Large-Scale Dataset for Training Complex Image-Text Models with Structural Annotations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zejian Li, Chenye Meng, Yize Li, Ling Yang, Shengyuan Zhang, Jiarui Ma, Jiayi Li, Guang Yang and 4 more
LAION-SG is a large-scale, high-quality dataset featuring scene graph annotations generated by GPT-4o for complex image-text pairs. Researchers from Zhejiang University, Alibaba Group, and Meta AI developed SDXL-SG, a foundation model leveraging this dataset to improve compositional image generation, achieving superior accuracy in capturing intricate object relationships on a new benchmark, CompSGen Bench.
Key insights
- Large-scale datasets with high-quality, structured scene graph annotations are crucial for training T2I models to accurately understand and generate complex compositional images.
- Leveraging multimodal large language models (LLMs) like GPT-4o with precise prompt engineering can effectively automate the generation of detailed scene graph annotations at scale, overcoming manual annotation bottlenecks.
- Integrating scene graph information through a lightweight GNN encoder within a strong T2I backbone like SDXL significantly enhances a model's ability to render intricate object relationships and attributes with minimal computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.