Reproduce “HumanSightBench: A Benchmark for Human Visual Perception in MLLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixin Su, Yangguang Lin, Zegang Peng, Yuntong Sun, Xuanyi Wang, Jiahao Wang, Qiankai Xu, Bing Yuan and 5 more
HumanSightBench evaluates whether multimodal language models recover ordinary visual facts on human-verified questions. Each item must have a clear, unique, image-grounded answer that three independent reviewers solve, and resist text-only shortcuts; model screening then removes already-solved items. Across 2,876 questions, GPT-6-Astra leads eight models at 65.6% pass@3, while Counting reaches only 27.5%. These scores describe retained questions, not everyday error prevalence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.