Reproduce “VIGOR: Benchmarking Visual Rationale Correctness in Multimodal Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanlin Chen, Tang Li, Xi Peng
VIGOR evaluates whether models’ attention maps for caption concepts align with pixel-level evidence masks across four concept levels. It compares Average Precision and Pointing Accuracy on fixed image–concept pairs, using attention as a proxy without additional fine-tuning. InternVL3.5-14B achieves highest AP, 31.2%, while InternVL3.5-38B has highest Pointing Accuracy, 38.7%, indicating limited alignment on this benchmark, not causal reliance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.