Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Time Series is Worth Five Experts: Heterogeneous Mixture of Experts for Traffic Flow Prediction

Guangyu Wang, Yujie Chen, Ming Gao, Zhiqiao Wu, Jiafu Tang, Jiabi Zhao

TL;DR

A Heterogeneous Mixture of Experts (TITAN) model for traffic flow prediction that combines variable-centric and prior knowledge-centric modeling techniques, and achieves improvements in all evaluation metrics, compared to previous state-of-the-art models.

Abstract

Accurate traffic prediction faces significant challenges, necessitating a deep understanding of both temporal and spatial cues and their complex interactions across multiple variables. Recent advancements in traffic prediction systems are primarily due to the development of complex sequence-centric models. However, existing approaches often embed multiple variables and spatial relationships at each time step, which may hinder effective variable-centric learning, ultimately leading to performance degradation in traditional traffic prediction tasks. To overcome these limitations, we introduce variable-centric and prior knowledge-centric modeling techniques. Specifically, we propose a Heterogeneous Mixture of Experts (TITAN) model for traffic flow prediction. TITAN initially consists of three experts focused on sequence-centric modeling. Then, designed a low-rank adaptive method, TITAN simultaneously enables variable-centric modeling. Furthermore, we supervise the gating process using a prior knowledge-centric modeling strategy to ensure accurate routing. Experiments on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate that TITAN effectively captures variable-centric dependencies while ensuring accurate routing. Consequently, it achieves improvements in all evaluation metrics, ranging from approximately 4.37\% to 11.53\%, compared to previous state-of-the-art (SOTA) models. The code is open at \href{https://github.com/sqlcow/TITAN}{https://github.com/sqlcow/TITAN}.

A Time Series is Worth Five Experts: Heterogeneous Mixture of Experts for Traffic Flow Prediction

TL;DR

A Heterogeneous Mixture of Experts (TITAN) model for traffic flow prediction that combines variable-centric and prior knowledge-centric modeling techniques, and achieves improvements in all evaluation metrics, compared to previous state-of-the-art models.

Abstract

Accurate traffic prediction faces significant challenges, necessitating a deep understanding of both temporal and spatial cues and their complex interactions across multiple variables. Recent advancements in traffic prediction systems are primarily due to the development of complex sequence-centric models. However, existing approaches often embed multiple variables and spatial relationships at each time step, which may hinder effective variable-centric learning, ultimately leading to performance degradation in traditional traffic prediction tasks. To overcome these limitations, we introduce variable-centric and prior knowledge-centric modeling techniques. Specifically, we propose a Heterogeneous Mixture of Experts (TITAN) model for traffic flow prediction. TITAN initially consists of three experts focused on sequence-centric modeling. Then, designed a low-rank adaptive method, TITAN simultaneously enables variable-centric modeling. Furthermore, we supervise the gating process using a prior knowledge-centric modeling strategy to ensure accurate routing. Experiments on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate that TITAN effectively captures variable-centric dependencies while ensuring accurate routing. Consequently, it achieves improvements in all evaluation metrics, ranging from approximately 4.37\% to 11.53\%, compared to previous state-of-the-art (SOTA) models. The code is open at \href{https://github.com/sqlcow/TITAN}{https://github.com/sqlcow/TITAN}.
Paper Structure (23 sections, 10 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 23 sections, 10 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. introduction
  2. Related Work
  3. Traffic Flow Prediction
  4. Mixture of Experts
  5. Methods
  6. Problem Definition
  7. Model Architecture
  8. Sequence-Centric Modeling
  9. variable-centered
  10. Annealing routing
  11. Experiment
  12. Experimental Settings
  13. Experimental Results
  14. Ablation Study
  15. Conclusion
  16. ...and 8 more sections

Figures (4)

  • Figure 1: TITAN Overview: Different color schemes represent three different modeling approaches: sequence-centric, variable-centric, and prior knowledge-centric.
  • Figure 2: TITAN model evaluation indicators under different parameter settings, marked in different colors
  • Figure 3: Box plots illustrating the error distributions of TITAN across all nodes for the Metr-LA and Pems-Bay datasets, segmented by prediction time steps.
  • Figure 4: Sensor distribution of the (a)METR-LA and (b)PEMS-BAY dataset.