Reproduce “From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiachun Li, Zhuoran Jin, Kang Liu, Jun Zhao
SpatialWalker tests spatial simulation: predicting how object relationships change as an observer and objects move, including intermediate states, without seeing intervening scenes. The authors train it in three stages and evaluate it on MentalGrid, a benchmark of simulated scenes. It scores 46.35 overall versus 12.54 for its Qwen2.5-VL-3B base, but real-world physical dynamics remain untested.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.