Reproduce “Overview of the NTCIR-19 SciClaimEval Task: Cross-modal Claim Verification in Scientific Papers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xanh Ho, André Greiner-Petter, Florian Boudin, Yun-Ang Wu, Sunisth Kumar, Tian Cheng Xia, Hiba Akbi, Tim Schopf and 1 more
SciClaimEval tests whether tables and figures support scientific claims by pairing each claim with original evidence and an altered version while keeping the wording fixed. It includes 1,633 samples from 180 papers across machine learning, natural language processing, and medicine. On PNG evidence, paired systems reached 98.1% pair accuracy, versus 73.8% without paired information; this does not establish standalone verification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.