Reproduce “AmQA: Amharic Question Answering Dataset” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tilahun Abedissa, Ricardo Usbeck, Yaregal Assabie
Researchers from Addis Ababa University and the University of Hamburg introduced AmQA, the first publicly available Amharic Question Answering dataset, comprising 2628 question-answer pairs derived from Amharic Wikipedia. Baseline models achieved an F1-score of 71.74 and an Exact Match of 50.76 using XLM-R_Large for reading comprehension, providing initial benchmarks for the language.
Key insights
- The AmQA dataset provides a foundational resource expected to stimulate new research in monolingual, cross-lingual, and open-domain Amharic QA.
- The challenge level of the AmQA dataset is comparable to that of existing multilingual QA datasets, as indicated by XLM-R_Large's performance.
- Even simple post-processing steps, such as character normalization, can measurably improve evaluation metrics for QA model outputs in low-resource language settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.