Reproduce “CIC: A Framework for Culturally-Aware Image Captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Youngsik Yun, Jihie Kim
A framework named Culturally-aware Image Captioning (CIC) generates image captions that explicitly describe cultural elements by leveraging Vision-Language Pre-trained Models and Large Language Models. Human evaluations indicate CIC captions are perceived as significantly more culturally descriptive than those from baseline models, and a new metric shows they contain a higher density of cultural information.
Key insights
- Integrating specific cultural details extracted via targeted VQA with a general image description allows Large Language Models to generate more comprehensive and culturally rich captions.
- Leveraging existing Vision-Language Pre-trained Models and Large Language Models through prompt engineering offers a resource-efficient alternative to creating new, culturally-annotated datasets.
- Human evaluation by participants from relevant cultural groups is critical for accurately assessing the quality of culturally-aware captions, as traditional metrics do not adequately capture cultural nuances.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.