Reproduce “AgentVQA: A Unified Benchmark for Agentic Visual Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu
AgentVQA introduces an offline multiple-choice benchmark for visual decisions across web interfaces, first-person video, robotics, games, and spatial planning. Its 1,000 curated questions use model-generated distractors and human verification. Gemini 3.7 Flash scores 58.5% accuracy. Spatial Planning rankings correlate with SpatialWorld’s Physical track at ρ = 0.895 across 12 models, but this narrow result does not establish universal interactive performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.