UrbanMind: Urban Dynamics Prediction with Multifaceted Spatial-Temporal Large Language Models

TL;DR

UrbanMind tackles robust urban dynamics forecasting under distributional shifts by unifying spatial-temporal data with large language models. It introduces Muffin-MAE for multifaceted fusion masked autoencoding, semantic-aware prompts with a Partially Frozen Attention fine-tuning scheme, and a test-time data reconstructor for fast domain adaptation. Across Shenzhen, Xi'an, and Chengdu, predicting speed, inflow, and demand, UrbanMind achieves state-of-the-art accuracy and robust zero-shot generalization, with ablations confirming the importance of each component. The work provides reproducible data and code, highlighting practical potential for real-world urban forecasting.

Abstract

Understanding and predicting urban dynamics is crucial for managing transportation systems, optimizing urban planning, and enhancing public services. While neural network-based approaches have achieved success, they often rely on task-specific architectures and large volumes of data, limiting their ability to generalize across diverse urban scenarios. Meanwhile, Large Language Models (LLMs) offer strong reasoning and generalization capabilities, yet their application to spatial-temporal urban dynamics remains underexplored. Existing LLM-based methods struggle to effectively integrate multifaceted spatial-temporal data and fail to address distributional shifts between training and testing data, limiting their predictive reliability in real-world applications. To bridge this gap, we propose UrbanMind, a novel spatial-temporal LLM framework for multifaceted urban dynamics prediction that ensures both accurate forecasting and robust generalization. At its core, UrbanMind introduces Muffin-MAE, a multifaceted fusion masked autoencoder with specialized masking strategies that capture intricate spatial-temporal dependencies and intercorrelations among multifaceted urban dynamics. Additionally, we design a semantic-aware prompting and fine-tuning strategy that encodes spatial-temporal contextual details into prompts, enhancing LLMs' ability to reason over spatial-temporal patterns. To further improve generalization, we introduce a test time adaptation mechanism with a test data reconstructor, enabling UrbanMind to dynamically adjust to unseen test data by reconstructing LLM-generated embeddings. Extensive experiments on real-world urban datasets across multiple cities demonstrate that UrbanMind consistently outperforms state-of-the-art baselines, achieving high accuracy and robust generalization, even in zero-shot settings.

Figures (5)

• Figure 1: Illustration of key challenges, including complex urban dynamics and distributional shifts.
• Figure 2: Masking strategies.
• Figure 3: UrbanMind Framework.
• Figure 4: Zero-shot Prediction: RMSE for traffic speed prediction over 4 hours in Shenzhen, Xi'an, and Chengdu.
• Figure 5: Impact of Hyperparameters on RMSE Performance for Traffic Speed Prediction in Xi'an.