Reproduce “WikiCausal: Corpus and Evaluation Framework for Causal Knowledge Graph Construction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Oktie Hassanzadeh

IBM Research developed WikiCausal, a publicly available corpus and evaluation framework for end-to-end causal knowledge graph construction from text, employing large language models for automated precision assessment. Preliminary experiments with the framework reveal that current methods struggle with instance-level causal relation extraction but can achieve high precision for class-level relations.

Key insights

  • End-to-end causal knowledge graph construction from text remains a highly challenging task for current state-of-the-art neural models, indicating a need for more sophisticated extraction methods.
  • Large Language Models can serve as a scalable and efficient alternative to human annotators for reliably evaluating the precision of extracted causal relations, despite some limitations with instance-level verification.
  • Current causal extraction methods exhibit a significant performance gap, demonstrating substantially better results at extracting class-level causal relations compared to instance-level relations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.