Unveiling the Inflexibility of Adaptive Embedding in Traffic Forecasting

TL;DR

This study evaluates state-of-the-art models on an extended traffic benchmark and observes substantial performance degradation in existing ST-GNNs over time, which is attributed to their limited inductive capabilities, and proposes a Principal Component Analysis embedding approach that enables models to adapt to new scenarios without retraining.

Abstract

Spatiotemporal Graph Neural Networks (ST-GNNs) and Transformers have shown significant promise in traffic forecasting by effectively modeling temporal and spatial correlations. However, rapid urbanization in recent years has led to dynamic shifts in traffic patterns and travel demand, posing major challenges for accurate long-term traffic prediction. The generalization capability of ST-GNNs in extended temporal scenarios and cross-city applications remains largely unexplored. In this study, we evaluate state-of-the-art models on an extended traffic benchmark and observe substantial performance degradation in existing ST-GNNs over time, which we attribute to their limited inductive capabilities. Our analysis reveals that this degradation stems from an inability to adapt to evolving spatial relationships within urban environments. To address this limitation, we reconsider the design of adaptive embeddings and propose a Principal Component Analysis (PCA) embedding approach that enables models to adapt to new scenarios without retraining. We incorporate PCA embeddings into existing ST-GNN and Transformer architectures, achieving marked improvements in performance. Notably, PCA embeddings allow for flexibility in graph structures between training and testing, enabling models trained on one city to perform zero-shot predictions on other cities. This adaptability demonstrates the potential of PCA embeddings in enhancing the robustness and generalization of spatiotemporal models.

Figures (7)

• Figure 1: Spatiotemporal analysis of urban development patterns in Sacramento, California (2016-2018). The sequence demonstrates the progressive intensification of building density (shown in blue) and its implications for transportation demand modeling. Red circles highlight key areas of urban transformation, indicating the dynamic nature of land use changes and their potential impact on travel demand patterns.
• Figure 2: We conducted a comparative analysis of LSTM against state-of-the-art (SOTA) models, including , STID STID, GWNet GWNet, AGCRN AGCRN, MTGNN wu2020connecting, TrendGCN jiang2023enhancing, and STAEformer STAEformer, assessing their performance on both in-distribution and out-of-distribution test datasets. In the in-distribution scenario, we utilized the original dataset, which had only a few weeks' interval from the training set, while the out-of-distribution scenario employed our newly collected dataset with a one-year interval. For models employing adaptive embedding, we conducted tests using both the original adaptive embedding and a zero embedding strategy, in which the trained adaptive embeddings were set to zero to reduce potential bias. Our results showed that LSTM maintained consistent performance across both in-distribution and out-of-distribution scenarios. This finding strongly suggests that the observed performance degradation in other models is primarily attributable to shifts in spatial relationships. We further observed that using zero embedding to mitigate bias improved test performance, though optimal results were not achieved.
• Figure 3: We found that adaptive embedding restricts the model to perform inference on the same graph, which is unrealistic in dynamic traffic scenarios due to the continuous development of cities. We propose a novel testing-time adaptive strategy that requires no additional training, where 'fire' indicates the optimization of target parameters and 'snowflake' represents the freezing of model parameters. Specifically, we apply PCA to compress the training input representations into a low-dimensional space. During testing, the same projection matrix is used to map the input into the same space. Within this new projected embedding, we can construct an entirely new graph structure, enabling the model to adapt to novel spatial relationships.
• Figure 4: Performance comparison of STID and AGCRN models with and without adaptive embedding across four datasets (METR-LA, PEMS-BAY, ER, and ETTm1). We observe that employing trainable adaptive embeddings results in excessive spatial distinguishability, leading to a performance degradation of the model compared to those without adaptive embeddings.
• Figure 5: We conducted a comparative analysis of model performance on the PEMS benchmark using PCA embeddings versus learnable embeddings. The MAE results reveal that employing PCA (frozen) embeddings does not deteriorate training outcomes. Surprisingly, in certain instances, it leads to substantially improved performance by mitigating over-fit issue.