Reproduce “From Show to Tell: A Survey on Deep Learning-based Image Captioning” on OpenResearch

About this paper

Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, Rita Cucchiara

Researchers from the University of Modena and Reggio Emilia and NVIDIA AI Technology Centre provide a comprehensive, updated survey of deep learning-based image captioning, systematically reviewing architectures, training strategies, datasets, and a wide array of evaluation metrics. The work includes a quantitative comparison of state-of-the-art methods and identifies key trends and future research directions.

Key insights

• Image captioning has shifted from global CNN features to fine-grained region-based features, culminating in the dominance of self-attentive Transformer architectures for both visual encoding and language generation.
• Large-scale vision-and-language pre-training, often combined with reinforcement learning for fine-tuning, has become the leading paradigm, significantly boosting model performance and generalization.
• While high performance on standard metrics is achievable, significant challenges persist in ensuring generalization to unseen objects and domains, generating diverse and non-boilerplate captions, and developing trustworthy AI solutions with fairness and bias mitigation.

Summary by alphaXiv.