Reproduce “RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Danni Xu, Shaojing Fan, Harry Cheng, Mohan Kankanhalli
RW-Post provides an auditable benchmark for multimodal fact-checking by linking social-media posts to structured reasoning and specific evidence. It evaluates models with no evidence, fixed evidence, or web-retrieved evidence. On its 1,268-instance test set, GPT-4o-mini’s Macro-F1 rises from 26.3% with post text alone to 67.8% with linked evidence; adding images did not consistently improve scores.
Key insights
- Reliable evidence is a dominant factor for accurate multimodal verification, with oracle evidence substantially improving model performance across the board.
- Current open-source large vision-language models (LVLMs) struggle to effectively integrate and leverage visual signals alongside textual evidence, often showing limited or negative performance changes with image input.
- Significant headroom exists for improving multimodal verification systems, particularly in faithful evidence grounding, robustly refuting misinformation, handling uncertainty, and generating reliably structured explanations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.