Reproduce “E-comIQ-ZH: A Human-Aligned Dataset and Benchmark for Fine-Grained Evaluation of E-commerce Posters with Chain-of-Thought” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Meiqi Sun, Mingyu Li, Junxiong Zhu

Researchers from Taobao & Tmall Group developed E-comIQ-ZH, a framework for evaluating AI-generated Chinese e-commerce posters, comprising a multi-dimensional dataset (E-comIQ-18k) and a specialized model (E-comIQ-M). This system provides fine-grained, human-aligned quality assessment and establishes a benchmark for generative models, improving evaluation accuracy for domain-specific visual content.

Key insights

  • A single holistic score is insufficient for e-commerce poster quality; multi-dimensional evaluation is crucial, with "Text" being a primary bottleneck for overall quality in Chinese posters.
  • Generic IQA models and MLLMs are inadequate for fine-grained e-commerce evaluation, especially for detecting subtle Chinese textual errors, underscoring the need for domain-specific adaptation.
  • OCR-based metrics are unreliable for detecting subtle visual artifacts in Chinese text rendering, emphasizing the value of human-aligned visual quality judgments for typographic correctness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.