Reproduce “UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark” on OpenResearch

About this paper

Zhaokun Zhou, Qiulin Wang, Bin Lin, Yiwei Su, Rui Chen, Xin Tao, Amin Zheng, Li Yuan and 2 more

Researchers from Peking University and Kuaishou Technology established UNIAA, a unified multi-modal image aesthetic assessment framework, by fine-tuning an MLLM with a novel data conversion paradigm. This framework achieved leading performance in aesthetic perception and zero-shot quantitative assessment, even surpassing GPT-4V in several perception metrics on its accompanying benchmark, UNIAA-Bench.

Key insights

• MLLMs can be effectively fine-tuned for complex aesthetic tasks, demonstrating superior performance in aesthetic perception, description, and zero-shot scoring when leveraging appropriately processed data.
• The IDCP is critical for democratizing MLLM-based aesthetic research by converting existing fragmented datasets into a unified, high-quality format, significantly reducing reliance on expensive manual annotation.
• Fine-tuning the vision encoder during aesthetic instruction tuning is beneficial for IAA, allowing the model to adapt its visual understanding to subtle aesthetic nuances and improving perception and assessment capabilities.

Summary by alphaXiv.