Reproduce “From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task” on OpenResearch

About this paper

Nicolas Martorell

Research explored how Large Language Models (LLMs) encode spatial information using a grid-world navigation task, identifying internal units that represent abstract spatial features like coordinates and boundaries, along with action correctness. The study found that spatial reasoning abilities scale with model size and that Cartesian input formats lead to superior performance.

Key insights

• LLMs demonstrate a clear scaling law for spatial navigation performance, with larger models achieving better success rates and path efficiency.
• The way spatial information is textually presented significantly influences LLM performance, with explicit Cartesian representations (like JSON) consistently outperforming other formats.
• Specific internal units within LLMs encode abstract spatial features (e.g., agent's X/Y coordinates, border presence) and action correctness, with some of these units showing invariance to diverse textual input formats and generalizing to unrelated spatial reasoning tasks.

Summary by alphaXiv.