Reproduce “Evaluation of Geolocation Capabilities of Multimodal Large Language Models and Analysis of Associated Privacy Risks” on OpenResearch

About this paper

Xian Zhang, Xiang Cheng

Researchers from Wuhan University's LIESMARS evaluated the inherent geolocation capabilities of state-of-the-art Multimodal Large Language Models using a global street-view image dataset, revealing that models like Google Gemini-2.5-Pro can achieve street-level accuracy in 49% of cases while posing substantial location privacy risks. The study elucidated a 'bimodal' reasoning pattern, where models either localize with high precision or fail broadly, and identified key visual cues like transit nodes and language as decisive for inference.

Key insights

• MLLMs exhibit a 'bimodal' distribution of localization errors, suggesting their reliance on identifying 'decisive evidence' rather than continuous human-like reasoning, leading to either very high accuracy or significant errors.
• MLLM geolocation reasoning follows a three-stage process: perception of visual and textual elements, analysis by correlating with internal world knowledge, and spatial verification using external geographic tools.
• Key visual cues such as transit nodes, language and text, and traffic signage act as 'decisive evidence' for MLLMs, enabling highly accurate location inference from images.

Summary by alphaXiv.