Reproduce “AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yipo Huang, Xiangfei Sheng, Zhichao Yang, Quan Yuan, Zhichao Duan, Pengfei Chen, Leida Li, Weisi Lin and 1 more
AesExpert introduces a multi-modality foundation model for image aesthetics perception, developed using AesMMIT, a novel instruction tuning dataset derived from 409,000 human aesthetic critiques across diverse image types. The model achieves leading performance on aesthetic assessment benchmarks, surpassing advanced commercial models like GPT-4V and Gemini Pro Vision by over 10% in overall score.
Key insights
- High-quality, human-centric aesthetic feedback is crucial for training MLLMs in subjective domains, outperforming larger, less focused datasets in aesthetic interpretation precision.
- Instruction tuning with diverse aesthetic questions and detailed human critiques effectively imbues MLLMs with a sophisticated understanding of elements like composition and emotional resonance.
- Grounding aesthetic learning in genuine human feedback significantly mitigates the hallucination problem in MLLMs, leading to more precise and human-aligned aesthetic interpretations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.