Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

City Foundation Models for Learning General Purpose Representations from OpenStreetMap

Pasquale Balsebre, Weiming Huang, Gao Cong, Yi Li

TL;DR

CityFM is presented, a self-supervised framework to train a foundation model within a selected geographical area that relies solely on open data from OSM, and produces multimodal representations, incorporating spatial, visual, and textual information.

Abstract

Pre-trained Foundation Models (PFMs) have ushered in a paradigm-shift in Artificial Intelligence, due to their ability to learn general-purpose representations that can be readily employed in a wide range of downstream tasks. While PFMs have been successfully adopted in various fields such as Natural Language Processing and Computer Vision, their capacity in handling geospatial data and answering urban questions remains limited. This can be attributed to the intrinsic heterogeneity of geospatial data, which encompasses different data types, including points, segments and regions, as well as multiple information modalities, such as a spatial position, visual characteristics and textual annotations. The proliferation of Volunteered Geographic Information initiatives, and the ever-increasing availability of open geospatial data sources, like OpenStreetMap, which is freely accessible globally, unveil a promising opportunity to bridge this gap. In this paper, we present CityFM, a self-supervised framework to train a foundation model within a selected geographical area of interest, such as a city. CityFM relies solely on open data from OSM, and produces multimodal representations of entities of different types, incorporating spatial, visual, and textual information. We analyse the entity representations generated using our foundation models from a qualitative perspective, and conduct quantitative experiments on road, building, and region-level downstream tasks. We compare its results to algorithms tailored specifically for the respective applications. In all the experiments, CityFM achieves performance superior to, or on par with, the baselines.

City Foundation Models for Learning General Purpose Representations from OpenStreetMap

TL;DR

CityFM is presented, a self-supervised framework to train a foundation model within a selected geographical area that relies solely on open data from OSM, and produces multimodal representations, incorporating spatial, visual, and textual information.

Abstract

Pre-trained Foundation Models (PFMs) have ushered in a paradigm-shift in Artificial Intelligence, due to their ability to learn general-purpose representations that can be readily employed in a wide range of downstream tasks. While PFMs have been successfully adopted in various fields such as Natural Language Processing and Computer Vision, their capacity in handling geospatial data and answering urban questions remains limited. This can be attributed to the intrinsic heterogeneity of geospatial data, which encompasses different data types, including points, segments and regions, as well as multiple information modalities, such as a spatial position, visual characteristics and textual annotations. The proliferation of Volunteered Geographic Information initiatives, and the ever-increasing availability of open geospatial data sources, like OpenStreetMap, which is freely accessible globally, unveil a promising opportunity to bridge this gap. In this paper, we present CityFM, a self-supervised framework to train a foundation model within a selected geographical area of interest, such as a city. CityFM relies solely on open data from OSM, and produces multimodal representations of entities of different types, incorporating spatial, visual, and textual information. We analyse the entity representations generated using our foundation models from a qualitative perspective, and conduct quantitative experiments on road, building, and region-level downstream tasks. We compare its results to algorithms tailored specifically for the respective applications. In all the experiments, CityFM achieves performance superior to, or on par with, the baselines.
Paper Structure (31 sections, 14 equations, 3 figures, 7 tables)

This paper contains 31 sections, 14 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Impact and Applications
  3. Predicted Impact
  4. Urban Applications
  5. Geospatial Data Management
  6. City Digital Twin
  7. Preliminaries
  8. CityFM
  9. Self-Supervised Learning
  10. Text-based Contrastive Objective
  11. Vision-Language Contrastive Objective
  12. Road-based Contrastive Objective
  13. Location Encoding
  14. Experiments
  15. Experimental Settings
  16. ...and 16 more sections

Figures (3)

  • Figure 1: CityFM's self-supervised pre-training framework. The three contrastive objectives are highlighted: (1) Text-based objective; (2) Vision-Language multimodal objective; (3) Road-based objective. Some dashed red lines are omitted for clarity.
  • Figure 2: Some examples demonstrating CityFM's capability to associate the visual characteristics of OpenStreetMap's polygons, with their corresponding functionality. We report the similarities of the shape and size with the textual encodings of the tags.
  • Figure 3: Top: All the road segments traversed by at least one bus loop, in OSM Singapore. Bottom: The same road segments, weighted by number of bus loops traversing them.