Reproduce “SKILL: Structured Knowledge Infusion for Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Fedor Moiseev, Zhe Dong, Enrique Alfonseca, Martin Jaggi

A new method, SKILL, directly infuses factual knowledge from structured knowledge graph triples into T5 models, bypassing the need for natural language conversion. This approach demonstrated improved factual recall on closed-book question answering tasks, performing comparably to text-based knowledge infusion while enhancing scalability.

Key insights

  • LLMs can effectively internalize and utilize factual knowledge directly from structured knowledge graph triples, performing comparably to or even slightly better than models trained on synthetic natural language representations of the same knowledge.
  • Direct knowledge infusion from KGs significantly enhances scalability and applicability by removing the complex and often unavailable requirement of aligning knowledge graph triples with natural language sentences.
  • The performance improvements from knowledge infusion are more pronounced in larger models, indicating that increased parameter capacity better leverages the extensive factual knowledge contained within large KGs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.