Reproduce “Enriching BERT with Knowledge Graph Embeddings for Document Classification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Malte Ostendorff, Peter Bourgonje, Maria Berger, Julian Moreno-Schneider, Georg Rehm, Bela Gipp
Researchers from DFKI GmbH and the University of Konstanz developed a method to classify German books using a German-specific BERT model enriched with structured metadata and Wikidata-derived author embeddings. This hybrid approach yielded an 87.20 micro-averaged F1-score for coarse-grained classification and 64.70 for detailed classification on short book blurbs, outperforming text-only BERT and traditional baselines.
Key insights
- Enriching pre-trained language models like BERT with structured external knowledge (knowledge graph embeddings) and tailored metadata consistently improves document classification performance, especially with limited textual input.
- Language-specific BERT models (e.g., BERT-German) significantly outperform multilingual BERT models for NLP tasks in their respective languages, indicating the value of domain-specific pre-training.
- The combination of deep neural networks with symbolic knowledge from knowledge graphs offers a robust approach for tackling complex, low-resource classification challenges and aiding content curation workflows.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.