Reproduce “Urban2Vec: Incorporating Street View Imagery and POIs for Multi-Modal Urban Neighborhood Embedding” on OpenResearch

About this paper

Zhecheng Wang, Haoyuan Li, Ram Rajagopal

The Urban2Vec framework generates compact, multi-modal vector representations of urban neighborhoods by integrating street view imagery and Point-of-Interest (POI) textual data through an unsupervised, two-stage learning process. These embeddings accurately predict demographic and socioeconomic attributes, as well as real estate values, demonstrating strong generalization across tasks and cities.

Key insights

• The combination of street view imagery (capturing physical environment) and POI textual data (reflecting business activity and social characteristics) provides a more comprehensive and robust characterization of urban neighborhoods than any single modality alone.
• Explicitly incorporating geospatial relationships among street view images via triplet loss is crucial for learning meaningful and context-aware visual embeddings that prevent isolated neighborhood clusters.
• Unsupervised multi-modal learning can generate highly effective and transferable neighborhood embeddings, achieving performance comparable to supervised methods on downstream tasks without the need for extensive labeled training data.

Summary by alphaXiv.