Reproduce “Overview of BioASQ 2025: The Thirteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara, Martin Krallinger, Miguel Rodríguez-Ortega, Eduard Rodriguez-López, Natalia Loukachevitch, Andrey Sakhovskiy and 12 more

The thirteenth BioASQ challenge expanded its scope to include multilingual and multi-document biomedical tasks, engaging 83 teams in advancing semantic indexing and question answering. This iteration generated new benchmark datasets and demonstrated progress in areas like clinical text summarization across languages, nested entity linking, and information extraction for specialized domains.

Key insights

  • While general Large Language Models (LLMs) are promising for simpler QA tasks, domain- and task-specific models (e.g., fine-tuned BERT-based architectures) often achieve superior performance for complex information extraction and linking challenges.
  • The expansion to multilingual and multi-document tasks highlights critical gaps in current AI capabilities for global healthcare, particularly for under-resourced languages and specialized clinical data.
  • Complex tasks, such as ternary mention-based relation extraction, remain highly challenging, indicating significant room for improvement in systems that simultaneously localize entities and identify specific relationships.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.