Reproduce “Communicating about Space: Language-Mediated Spatial Integration Across Partial Views” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ankur Sikarwar, Debangan Mishra, Sudarshan Nikhil, Ponnurangam Kumaraguru, Aishwarya Agrawal
This research introduces COSMIC, a benchmark evaluating Multimodal Large Language Models (MLLMs) on language-mediated spatial integration from partial, egocentric views. MLLMs show a significant performance gap compared to humans (best model at 71.82% vs. humans at 95.22%), particularly struggling with higher-level spatial reasoning and cross-view grounding.
Key insights
- MLLMs exhibit a consistent hierarchical degradation in performance across spatial reasoning tasks, with significant struggles in perspective-taking and nearly failing at full cognitive mapping from partial views.
- The act of communication itself introduces substantial difficulty for MLLMs, as performance drops significantly when requiring dialogue compared to seeing both views simultaneously.
- Failure analysis reveals dominant issues in cross-view grounding (referential ambiguity, instance merging/duplication) and geometric/relational reasoning, rather than primarily perceptual errors, suggesting deeper deficits in spatial integration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.