Reproduce “Exploring LLM Capabilities in Extracting DCAT-Compatible Metadata for Data Cataloging” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lennart Busch, Daniel Tebernum, Gissel Velarde
Researchers at IU International University of Applied Sciences and Fraunhofer ISST systematically evaluated various Large Language Models (LLMs) for extracting DCAT-compatible metadata from text, demonstrating that LLMs can achieve human-level accuracy, with fine-tuning significantly boosting classification performance and larger models generally outperforming smaller ones, though smaller models are efficient for basic tasks.
Key insights
- Autoregressive LLMs can generate descriptive, DCAT-compatible metadata with accuracy comparable to human-curated metadata, especially with few-shot prompting.
- Fine-tuning LLMs for specific classification tasks (like dcat:theme) provides superior accuracy, outperforming few-shot prompting in these scenarios.
- Larger LLMs generally deliver higher quality and more consistent metadata, particularly for complex semantic tasks, while smaller models remain efficient for simpler information extraction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.