Reproduce “Affect Recognition in Ads with Application to Computational Advertising” on OpenResearch

About this paper

Abhinav Shukla, Shruti Shriya Gullapuram, Harish Katti, Karthik Yadati, Mohan Kankanhalli, Ramanathan Subramanian

Researchers from the International Institute of Information Technology, Hyderabad, and collaborators created a validated dataset of commercial advertisements labeled for emotional content and developed a deep learning framework, AdAffectNet, for affect recognition. This framework, which uses domain-adapted CNN features, achieved an F1-score of 0.96 for valence recognition and demonstrated that improved affect prediction significantly enhances subjective user experience in computational advertising's ad insertion.

Key insights

• A consistently labeled dataset of advertisements reveals that affective dimensions (arousal-valence) are largely uncorrelated and uniformly distributed, departing from typical 'C'-shape patterns, indicating ads' specific emotional messaging goals.
• Deep learning-based AdAffectNet features significantly outperform traditional low-level audio-visual features for ad affect recognition, with multi-task learning further boosting performance to F1-scores of 0.96 for valence.
• Improving affect prediction, specifically valence, through AAN leads to a measurably better subjective user experience in computational advertising by enabling contextually appropriate ad insertions, suggesting that contextual relevance is a crucial metric beyond mere recall.

Summary by alphaXiv.