Reproduce “Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiwan Chung, Sungjae Lee, Minseo Kim, Seungju Han, Ashkan Yousefpour, Jack Hessel, Youngjae Yu
Researchers from Yonsei University and collaborators introduced VisArgs, a new dataset and benchmark to evaluate AI's capacity for visual argument understanding. Experiments on VisArgs show that current multimodal AI models struggle with "selective vision," identifying argumentatively relevant visual cues (e.g., GPT-4-O at 79.50% accuracy compared to human 98.00% for premise identification), while basic visual grounding is more robust.
Key insights
- "Selective vision" – the ability to identify and focus on argumentatively relevant visual cues – is the primary bottleneck for AI in understanding visual arguments.
- True comprehension of visual arguments necessitates discerning the contextual relevance of visual stimuli, rather than mere perceptual recognition.
- Current AI models often rely on common word associations rather than sophisticated reasoning about visual object relevance in an argumentative context.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.