Reproduce “EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huanhuan Ma, Weizhi Xu, Yifan Wei, Liuji Chen, Liang Wang, Qiang Liu, Shu Wu
A new dataset, EX-FEVER, enables multi-hop explainable fact verification by providing over 60,000 human-annotated claims with detailed natural language explanations. Developed by researchers at the University of Chinese Academy of Sciences and ByteDance Inc., this dataset facilitates the development of systems that can process complex, real-world claims and also highlights that large language models can generate explanations that outperform fine-tuned models like BART and function effectively as reasoning planners.
Key insights
- Document retrieval remains a significant bottleneck in the end-to-end fact verification pipeline, as the quality of retrieved evidence directly impacts subsequent explanation generation.
- The human-annotated golden explanations in EX-FEVER are of high quality and utility, as evidenced by a BERT-based model achieving near-optimum accuracy (99.29%) when trained with these explanations.
- Large Language Models (LLMs) like ChatGPT demonstrate strong capabilities as "planners" for reasoning steps and as generators of high-quality explanations, even outperforming fine-tuned models like BART in ROUGE scores when given few-shot examples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.