Reproduce “An Empirical Study of Recent Large Language Models for Knowledge Graph to Text Generation on Standard Benchmarks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Priyanka, Renaud Delbru, Souhail Bakkali, Mickaël Coustaty, Antoine Doucet

The paper compares prompted language models with fine-tuned T5-large for knowledge-graph-to-text generation across three benchmarks. It evaluates outputs using reference-based metrics and graph-grounded judges. On WebNLG17, GPT-5-mini scores 51.94 BLEU versus T5-large’s 58.57. Judges generally score prompted models higher on faithfulness and coverage, but often miss unsupported elaborations; fine-tuned errors rise faster with graph size in this setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.