Reproduce “SciREX: A Challenge Dataset for Document-Level Information Extraction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy
SciREX introduces a full-document benchmark and neural baseline for extracting dataset, metric, task, and method tuples from scientific papers. Its pipeline links mentions across sections into result relations. 99% of 4-ary relations span sentences and 55% span sections; relation F1 rises from 0.008 to 0.268 with gold salient-cluster information, indicating a bottleneck without showing that the baseline solves extraction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.