Reproduce “Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mi Luo, Rohan Dugad, Zihui Xue, Alex Dimakis, Kristen Grauman

The authors probe four latent-reasoning models by perturbing visual states or reasoning text. Visual perturbations usually change answer probability less than text perturbations, a mismatch they call Mental Simulation Collapse. They introduce AnchorSim, which contrasts full and visual-blind predictions during training. Its average across nine benchmark entries is 62.4%, versus 48.9% for the Qwen2.5-VL-7B base, at 7B scale.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.