Reproduce “VieCap4H-VLSP 2021: ObjectAoA-Enhancing performance of Object Relation Transformer with Attention on Attention for Vietnamese image captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nghia Hieu Nguyen, Duong T.D. Vo, Minh-Quan Ha
Researchers from the University of Information Technology and Vietnam National University, Ho Chi Minh City, developed ObjectAoA, a Transformer-based architecture that integrates the Attention on Attention (AoA) mechanism into the Object Relation Transformer (ORT). This model consistently improved image captioning performance for Vietnamese, achieving a 0.126 CIDEr score increase on the UIT-ViIC dataset and winning the VLSP Image Captioning Shared Task 2021 on the VieCap4H dataset.
Key insights
- The integration of AoA effectively enhances the correlation between the output of geometric attention and query vectors, improving the model's ability to integrate visual and linguistic features more clearly.
- Combining geometric-aware object attention with an output-input correlation mechanism allows for more precise object understanding and the generation of contextually richer and more specific captions.
- Advanced Transformer-based architectures, when appropriately adapted, can significantly improve image captioning performance for low-resource languages like Vietnamese, addressing unique linguistic features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.