Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Frequency Enhanced Pre-training for Cross-city Few-shot Traffic Forecasting

Zhanyu Liu, Jianrong Ding, Guanjie Zheng

TL;DR

FEPCross addresses data scarcity in developing cities for traffic forecasting by exploiting cross-city spectral similarity. It introduces a frequency-enhanced pre-training framework with a Cross-Domain Spatial-Temporal Encoder that processes time, amplitude, and phase domains via Fourier transforms, trained with reconstruction and contrastive losses. In fine-tuning, it enriches few-shot data with masking-reconstruction and maintains a momentum-updated meta-graph to stabilize transfers, then uses a Graph Wavenet-based ST-model for forecasting. Experiments on four real-world datasets show FEPCross consistently outperforms diverse baselines, demonstrating effective spectral knowledge transfer and robustness in cross-city settings. This work advances ITS capability for data-poor cities by leveraging spectral patterns for cross-city transfer.

Abstract

The field of Intelligent Transportation Systems (ITS) relies on accurate traffic forecasting to enable various downstream applications. However, developing cities often face challenges in collecting sufficient training traffic data due to limited resources and outdated infrastructure. Recognizing this obstacle, the concept of cross-city few-shot forecasting has emerged as a viable approach. While previous cross-city few-shot forecasting methods ignore the frequency similarity between cities, we have made an observation that the traffic data is more similar in the frequency domain between cities. Based on this fact, we propose a \textbf{F}requency \textbf{E}nhanced \textbf{P}re-training Framework for \textbf{Cross}-city Few-shot Forecasting (\textbf{FEPCross}). FEPCross has a pre-training stage and a fine-tuning stage. In the pre-training stage, we propose a novel Cross-Domain Spatial-Temporal Encoder that incorporates the information of the time and frequency domain and trains it with self-supervised tasks encompassing reconstruction and contrastive objectives. In the fine-tuning stage, we design modules to enrich training samples and maintain a momentum-updated graph structure, thereby mitigating the risk of overfitting to the few-shot training data. Empirical evaluations performed on real-world traffic datasets validate the exceptional efficacy of FEPCross, outperforming existing approaches of diverse categories and demonstrating characteristics that foster the progress of cross-city few-shot forecasting.

Frequency Enhanced Pre-training for Cross-city Few-shot Traffic Forecasting

TL;DR

FEPCross addresses data scarcity in developing cities for traffic forecasting by exploiting cross-city spectral similarity. It introduces a frequency-enhanced pre-training framework with a Cross-Domain Spatial-Temporal Encoder that processes time, amplitude, and phase domains via Fourier transforms, trained with reconstruction and contrastive losses. In fine-tuning, it enriches few-shot data with masking-reconstruction and maintains a momentum-updated meta-graph to stabilize transfers, then uses a Graph Wavenet-based ST-model for forecasting. Experiments on four real-world datasets show FEPCross consistently outperforms diverse baselines, demonstrating effective spectral knowledge transfer and robustness in cross-city settings. This work advances ITS capability for data-poor cities by leveraging spectral patterns for cross-city transfer.

Abstract

The field of Intelligent Transportation Systems (ITS) relies on accurate traffic forecasting to enable various downstream applications. However, developing cities often face challenges in collecting sufficient training traffic data due to limited resources and outdated infrastructure. Recognizing this obstacle, the concept of cross-city few-shot forecasting has emerged as a viable approach. While previous cross-city few-shot forecasting methods ignore the frequency similarity between cities, we have made an observation that the traffic data is more similar in the frequency domain between cities. Based on this fact, we propose a \textbf{F}requency \textbf{E}nhanced \textbf{P}re-training Framework for \textbf{Cross}-city Few-shot Forecasting (\textbf{FEPCross}). FEPCross has a pre-training stage and a fine-tuning stage. In the pre-training stage, we propose a novel Cross-Domain Spatial-Temporal Encoder that incorporates the information of the time and frequency domain and trains it with self-supervised tasks encompassing reconstruction and contrastive objectives. In the fine-tuning stage, we design modules to enrich training samples and maintain a momentum-updated graph structure, thereby mitigating the risk of overfitting to the few-shot training data. Empirical evaluations performed on real-world traffic datasets validate the exceptional efficacy of FEPCross, outperforming existing approaches of diverse categories and demonstrating characteristics that foster the progress of cross-city few-shot forecasting.
Paper Structure (13 sections, 29 equations, 5 figures, 3 tables)

This paper contains 13 sections, 29 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Method
  5. Pre-Training
  6. Fine-Tuning
  7. Experiment
  8. Experiment Settings
  9. RQ1: Overall Performance
  10. RQ2: Ablation Study
  11. RQ3: Reconstruction Analysis
  12. RQ4: Cross Domain Analysis
  13. Conclusion

Figures (5)

  • Figure 1: (a) The mean cosine similarity of one-week traffic speed data within the time and frequency domains across four cities. We can observe a higher level of similarity in the frequency domain. (b) An illustrative instance showcasing a pair of data in both the time domain and frequency domain reflects that, despite containing identical data, the frequency domain exhibits a significantly higher level of cosine similarity.
  • Figure 2: Overall Framework of FEPCross.
  • Figure 3: Illustration of Training Data Enriching Module.
  • Figure 4: Reconstruction Analysis. The MAE of the reconstruction on the time domain is shown. Pretrain-base and Pretrain base+F have the same performance since they both individually reconstruct the data of the time domain. FEPCross, i.e., Pretrain base+F+D+S+C, is the final version.
  • Figure 5: The attention map to reconstruct the time domain.