Reproduce “Large Language Models as Oracles for Ontology Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sviatoslav Lushnei, Dmytro Shumskyi, Severyn Shykula, Ernesto Jimenez-Ruiz, Artur d'Avila Garcez
Lushnei et al. from Ukrainian Catholic University and City St George's, University of London, integrated Large Language Models (LLMs) as 'Oracles' within the LogMap ontology alignment system to validate uncertain mappings. This hybrid approach significantly improved LogMap's F-score across OAEI Bio-ML tasks, achieving a top-2 overall rank with an F-score of 0.751 in the OAEI 2025 campaign, by selectively applying LLM reasoning to challenging cases.
Key insights
- LLMs demonstrate strong diagnostic capabilities for complex, uncertain ontology mappings where traditional systems falter.
- A targeted 'LLM-in-the-loop' strategy, focusing LLM resources only on challenging cases, offers a practical and cost-effective approach to improve alignment quality.
- Carefully crafted, ontology-driven prompts, incorporating lexical and hierarchical context, are essential for eliciting accurate reasoning from LLMs in knowledge-intensive tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.