Reproduce “UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment” on OpenResearch

About this paper

Hantao Zhou, Longxiang Tang, Rui Yang, Guanyi Qin, Yan Zhang, Yutao Li, Xiu Li, Runze Hu and 1 more

UniQA presents a unified vision-language pre-training framework for assessing image quality and aesthetics, utilizing MLLM-assisted caption generation and a data purification strategy. The model achieves state-of-the-art performance across seven IQA and three IAA benchmarks, showing an SRCC of 0.940 on KADID for IQA and 0.776 on AVA for IAA, and exhibits strong generalization to few-label and diverse downstream tasks.

Key insights

• Human perception of image quality and aesthetics is inherently interconnected, allowing a single foundational model to learn mutually beneficial representations for both tasks.
• MLLMs can effectively generate high-quality, relevant text descriptions for image assessment when guided by Mean Opinion Scores (MOS) and task-specific prompts.
• The combination of aesthetics relevance and informativeness is crucial for refining noisy human-annotated captions, improving the quality of text data for vision-language model training.
• Unified pre-training on diverse data significantly enhances commonsense knowledge, enabling the model to focus on quality- and aesthetics-relevant regions simultaneously and generalize well across tasks.

Summary by alphaXiv.