Reproduce “How AI Experiences Art: Emergent Aesthetic Structure in a Self-Supervised Multimodal Embedding Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Corey D.C. Heath
An independent researcher developed a self-supervised multimodal framework to explore how AI models autonomously categorize human-produced media based on aesthetic qualities across text, audio, image, and video. The framework clustered diverse media into 28 distinct aesthetic concepts, revealing AI-discovered dimensions like "Timeless-Nostalgia" and "Soulful-Contemplation" that transcend traditional human aesthetic categories.
Key insights
- AI can autonomously organize multimodal data into coherent aesthetic structures, yielding a finer granularity (28 clusters) and internal homogeneity compared to human-aligned categories (6 registers).
- AI's aesthetic categorization quantitatively diverges from human labels (Adjusted Rand Index of 0.15), suggesting the discovery of distinct underlying structural principles for organizing artistic content.
- Emergent AI concepts like "Timeless-Nostalgia" and "Soulful-Contemplation" cut across traditional human aesthetic registers, indicating that AI can identify aesthetic dimensions that traverse emotional categories rather than merely media boundaries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.