Reproduce “Pathways of Visual Information Flow in Vision-Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Israfel Salazar, Stella Frank, Dan Oneata, Desmond Elliott, Constanza Fierro

The authors use causal patching to trace how vision-language models carry image information to answers. They compare image-token representations with question-text representations across recognition, spatial relations, and localization, and test how task, image distribution, and prompt format affect the route. For Qwen3-VL-4B, 98% of 200 synthetic-relation predictions followed patched text information; this measures source alignment, not accuracy, and the route varies across settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.