Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models

Weijia Zhang, Jindong Han, Zhao Xu, Hang Ni, Tengfei Lyu, Hao Liu, Hui Xiong

TL;DR

This paper defines Urban General Intelligence (UGI) and Urban Foundation Models (UFMs), arguing that UFMs can autonomously understand and manage complex urban systems by fusing multi-source, multi-granularity, and multimodal urban data. It provides a data-centric taxonomy of UFMs across seven modalities (language, vision, time series, trajectory, geovector, multimodal, and others) and outlines a framework for building versatile UFMs, including data integration, spatio-temporal reasoning, and governance. The work surveys existing UFMs, detailing both unimodal and cross-modal pre-training and adaptation techniques, and discusses numerous applications in transportation, urban planning, energy, and public safety. Finally, it presents a forward-looking research agenda emphasizing spatio-temporal reasoning, multi-task AI agents, privacy and security, and data- and model-pricing considerations to enable scalable, responsible deployment in smart cities.

Abstract

The integration of machine learning techniques has become a cornerstone in the development of intelligent urban services, significantly contributing to the enhancement of urban efficiency, sustainability, and overall livability. Recent advancements in foundational models, such as ChatGPT, have introduced a paradigm shift within the fields of machine learning and artificial intelligence. These models, with their exceptional capacity for contextual comprehension, problem-solving, and task adaptability, present a transformative opportunity to reshape the future of smart cities and drive progress toward Urban General Intelligence (UGI). Despite increasing attention to Urban Foundation Models (UFMs), this rapidly evolving field faces critical challenges, including the lack of clear definitions, systematic reviews, and universalizable solutions. To address these issues, this paper first introduces the definition and concept of UFMs and highlights the distinctive challenges involved in their development. Furthermore, we present a data-centric taxonomy that classifies existing research on UFMs according to the various urban data modalities and types. In addition, we propose a prospective framework designed to facilitate the realization of versatile UFMs, aimed at overcoming the identified challenges and driving further progress in this field. Finally, this paper explores the wide-ranging applications of UFMs within urban contexts, illustrating their potential to significantly impact and transform urban systems. A comprehensive collection of relevant research papers and open-source resources have been collated and are continuously updated at: https://github.com/usail-hkust/Awesome-Urban-Foundation-Models.

Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models

TL;DR

This paper defines Urban General Intelligence (UGI) and Urban Foundation Models (UFMs), arguing that UFMs can autonomously understand and manage complex urban systems by fusing multi-source, multi-granularity, and multimodal urban data. It provides a data-centric taxonomy of UFMs across seven modalities (language, vision, time series, trajectory, geovector, multimodal, and others) and outlines a framework for building versatile UFMs, including data integration, spatio-temporal reasoning, and governance. The work surveys existing UFMs, detailing both unimodal and cross-modal pre-training and adaptation techniques, and discusses numerous applications in transportation, urban planning, energy, and public safety. Finally, it presents a forward-looking research agenda emphasizing spatio-temporal reasoning, multi-task AI agents, privacy and security, and data- and model-pricing considerations to enable scalable, responsible deployment in smart cities.

Abstract

The integration of machine learning techniques has become a cornerstone in the development of intelligent urban services, significantly contributing to the enhancement of urban efficiency, sustainability, and overall livability. Recent advancements in foundational models, such as ChatGPT, have introduced a paradigm shift within the fields of machine learning and artificial intelligence. These models, with their exceptional capacity for contextual comprehension, problem-solving, and task adaptability, present a transformative opportunity to reshape the future of smart cities and drive progress toward Urban General Intelligence (UGI). Despite increasing attention to Urban Foundation Models (UFMs), this rapidly evolving field faces critical challenges, including the lack of clear definitions, systematic reviews, and universalizable solutions. To address these issues, this paper first introduces the definition and concept of UFMs and highlights the distinctive challenges involved in their development. Furthermore, we present a data-centric taxonomy that classifies existing research on UFMs according to the various urban data modalities and types. In addition, we propose a prospective framework designed to facilitate the realization of versatile UFMs, aimed at overcoming the identified challenges and driving further progress in this field. Finally, this paper explores the wide-ranging applications of UFMs within urban contexts, illustrating their potential to significantly impact and transform urban systems. A comprehensive collection of relevant research papers and open-source resources have been collated and are continuously updated at: https://github.com/usail-hkust/Awesome-Urban-Foundation-Models.
Paper Structure (57 sections, 3 equations, 3 figures)

This paper contains 57 sections, 3 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Basics of Urban Foundation Models
  3. Data Characteristics
  4. Key Techniques of Building Urban Foundation Models
  5. Pre-training
  6. Adaptation
  7. Challenges of Building Urban Foundation Models
  8. Overview of Urban Foundation Models
  9. Language-based Models
  10. Unimodal Pre-training
  11. Unimodal Adaptation
  12. Vision-based Models
  13. Unimodal Pre-training
  14. Unimodal Adaptation
  15. Time Series-based Models
  16. ...and 42 more sections

Figures (3)

  • Figure 1: Urban Foundation Models (UFMs) are pre-trained on multi-source, multi-granularity and multimodal urban data and can be adapted for a variety of downstream urban application domains.
  • Figure 2: A data-centric taxonomy for existing UFMs-related works based on the types of urban data modalities.
  • Figure 3: A prospective framework for building versatile UFMs.

Theorems & Definitions (1)

  • Definition 1: Urban Foundation Models