Reproduce “Afri-MCQA: Multimodal Cultural Question Answering for African Languages” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Muhidin A. Mohamed, Debela Desalegn Yadeta, Negasi Haile Abadi and 11 more
Afri-MCQA introduces the first large-scale, multilingual, multimodal benchmark for African cultural visual question answering, encompassing 15 languages with over 7.5k text and speech Q&A pairs. The benchmark demonstrates that current MLLMs struggle with African cultural understanding, particularly with native language speech queries, often due to foundational speech processing limitations, while proprietary models exhibit more robust performance.
Key insights
- Current MLLMs, especially open-weight variants, exhibit substantial limitations in interpreting African cultural contexts, with performance significantly deteriorating when queries are posed in native African languages or through the speech modality.
- The inability of open-weight models to perform basic speech processing tasks like Language Identification (LID) and Automatic Speech Recognition (ASR) for African languages constitutes a critical bottleneck, hindering their ability to engage in meaningful cultural reasoning from audio inputs.
- There is a distinct gap between MLLMs' general factual knowledge and their understanding of Africa-specific cultural nuances, suggesting that models require explicit pretraining on diverse African cultural content beyond linguistic data alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.