Reproduce “Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Henglin Liu, Nisha Huang, Chang Liu, Jiangpeng Yan, Huijuan Huang, Jixuan Ying, Tong-Yee Lee, Pengfei Wan and 1 more
ArtQuant is a Multimodal Large Language Model framework that quantifies artistic image aesthetics by leveraging hierarchical descriptions generated through an iterative LLM-assisted pipeline, creating a 70,000-entry Refined Aesthetic Description (RAD) dataset. This approach leads to superior aesthetic assessment performance across multiple datasets compared to existing generalist and specialist models, alongside marked improvements in training efficiency.
Key insights
- Hierarchical descriptive texts, spanning perception, cognition, and emotion, serve as effective carriers of aesthetic prior knowledge for MLLMs, enabling a more comprehensive understanding of artistic elements.
- An information-theoretic analysis demonstrates that improvements in description sufficiency and the model's ability to generate accurate descriptions directly reduce the uncertainty in aesthetic score prediction.
- A progressive training strategy, combining Multi-Task Aesthetic Training (MAT) and Score-Only Training (SOT), efficiently specializes MLLMs for complex aesthetic tasks by first establishing hierarchical semantic grounding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.