Reproduce “AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tobi Olatunji, Charles Nimo, Abraham Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Chinemelu Aka, Folafunmi Omofoye and 18 more
AfriMed-QA introduces an English-language Pan-African benchmark of 15,275 medical questions sourced from more than 60 medical schools across 16 countries and covering 32 specialties. The authors evaluate 30 large language models. GPT-4o achieves 0.8814 accuracy on MedQA and 0.7928 on AfriMed-QA Expert MCQs, a gap between distinct question sets that does not isolate geography as its cause.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.