Reproduce “Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoxing Lian, Aidong Yang, Jun Zhu, Peng Wang, Yue Zhang
Researchers at Telco AI Labs introduced SpatiaLite, a novel synthetic benchmark designed to evaluate both spatial reasoning accuracy and efficiency (token cost) in Vision-Language Models. The study reveals that even advanced VLMs struggle significantly with visual-centric spatial reasoning tasks and exhibit profound inefficiency, while their proposed Imagery-Driven Framework enhanced an open-source VLM's spatial reasoning by implicitly constructing an internal spatial world model.
Key insights
- Advanced VLMs, including proprietary models, perform poorly on visual-centric spatial reasoning tasks (e.g., Mental Rotation, Gemini 2.5 Pro scored 20.5% vs. human 100%), indicating a fundamental deficit in visuospatial imagination and a reliance on linguistic strategies.
- VLMs exhibit profound reasoning inefficiency, with token usage increasing exponentially with task complexity, often generating over 10,000 tokens for hard problems due to perception, transformation, or strategic planning errors.
- The Imagery-Driven Framework (IDF) effectively constructs an implicit internal spatial world model, leading to improved spatial reasoning performance (e.g., Rubik's Cube accuracy for Qwen-2.5-VL-7B increased from 20.1% to 44.7%) by enhancing understanding of spatial transformations and board layout predictions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.