Reproduce “VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lingxiao Li, Yifan Wang, Xinyan Gao, Chen Tang, Xiangyu Yue, Chenyu You
VisReason introduces a large-scale dataset for visual Chain-of-Thought (CoT) reasoning, including a depth-augmented subset called VisReason-Pro, to train Multimodal Large Language Models (MLLMs) with spatially grounded, multi-round reasoning. MLLMs fine-tuned on VisReason achieved an average score of 0.802 on the Visual-CoT benchmark, outperforming leading MLLMs, with substantial improvements in fine-grained recognition and spatial relational reasoning, and demonstrated enhanced RoI localization and depth-aware grounding.
Key insights
- Providing multi-round, spatially grounded Chain-of-Thought (CoT) supervision with explicit RoI bounding boxes guides MLLMs towards a human-like global-to-local 'zoom-and-verify' reasoning process.
- Integrating pseudo-3D cues (monocular depth and instance segmentation) into CoT rationales enhances MLLMs' ability to perform more robust ordinal-depth and occlusion-aware spatial reasoning.
- The effectiveness of CoT reasoning in MLLMs is sensitive to evaluation protocols; while multi-round inference improves fine-grained tasks, it can show reduced performance on benchmarks designed for direct QA if the evaluation format is mismatched.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.