Reproduce “RadGraph: Extracting Clinical Entities and Relations from Radiology Reports” on OpenResearch

About this paper

Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang and 4 more

RadGraph introduces a novel information extraction schema and meticulously annotated datasets for extracting fine-grained clinical entities and their relations from radiology reports. The work establishes a deep learning benchmark, achieving a micro F1 of 0.940 for named entity recognition and 0.823 for relation extraction on a test set, and releases a large inference dataset linked to chest radiographs.

Key insights

• The designed schema achieved high coverage, annotating an average of 87.7% of sentences and 46.4% of tokens in the Findings and Impression sections of development reports.
• Annotation ambiguity, particularly for 'Observation: Uncertain' entities and 'Suggestive Of' relations, and for reports from the CheXpert dataset, resulted in significant inter-observer variability.
• The DYGIE++ model with PubMedBERT initializations consistently delivered the highest performance, demonstrating the effectiveness of joint extraction frameworks and specialized biomedical language models for this task.

Summary by alphaXiv.