Reproduce “DetailVerifyBench: A Benchmark for Dense Hallucination Localization in Long Image Captions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinran Wang, Yuxuan Zhang, Xiao Zhang, Haolong Yan, Muxi Diao, Songyu Xu, Zhonghao Yan, Hongbing Li and 2 more
DetailVerifyBench measures whether models can localize hallucinated words in long image captions. It covers 1,000 images across five domains, with token-level labels for real and synthetic errors. Gemini-3-Pro-Preview achieves token-level F1 scores of 0.64 on synthetic errors and 0.10 on real errors, showing a gap on this benchmark, not reliable real-world performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.