Reproduce “Text Generation from Knowledge Graphs with Graph Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, Hannaneh Hajishirzi

Researchers from the University of Washington and University of Edinburgh developed GraphWriter, a graph-transformer-based encoder-decoder model for generating multi-sentence scientific abstracts from automatically extracted knowledge graphs. The model achieved state-of-the-art BLEU and METEOR scores, and human evaluations confirmed superior coherence and informativeness compared to baseline methods.

Key insights

  • The novel Graph Transformer encoder, by augmenting local self-attention with block networks for global contextualization, effectively captures long-distance dependencies and overall structural patterns in knowledge graphs.
  • Automatically extracted knowledge graphs, even with inherent noise, can serve as a viable and scalable input for generating high-quality, coherent multi-sentence text.
  • Explicitly modeling the relational structure of knowledge graphs, rather than treating entities as an unstructured list, significantly improves the generated text's coherence, grammar, and overall quality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.