Reproduce “When Plausible Is Not Realistic: Evaluating Human Mobility in LLM-Based Urban Simulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gustavo H. Santos, Aline Carneiro Viana, Thiago H. Silva
The paper evaluates mobility realism in AgentSociety and a reimplemented CitySim by comparing simulated trajectories with observed data across spatial laws, temporal rhythms, movement motifs, behavioral profiles, and semantic activities. In Greater Paris, travel-distance Wasserstein distance is 14.83 ± 0.35 kilometers for AgentSociety versus 7.53 ± 1.67 for CitySim, but neither system consistently matches spatial flows and timing.
Key insights
- Generating plausible mobility narratives and schedules with LLMs does not ensure adherence to empirically realistic human mobility patterns or statistical regularities.
- LLM-driven simulators exhibit systematic biases and quantitative discrepancies across multiple dimensions of human mobility, including spatial, temporal, and topological characteristics.
- Increased complexity in LLM-driven planning or POI selection does not inherently improve mobility realism and can sometimes degrade temporal dynamics or lead to unstable agent behaviors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.