HiMoE: Heterogeneity-Informed Mixture-of-Experts for Fair Spatial-Temporal Forecasting

TL;DR

HiMoE tackles the dual challenge of accuracy and fairness in spatial-temporal forecasting under node heterogeneity. It introduces HiGCN to fuse static and dynamic node relations for trend heterogeneity and NMoE to route nodes to specialized experts for cardinality heterogeneity, coupled with STFairBench to guide fairness during training and evaluation. The approach achieves state-of-the-art results across four real-world datasets and significantly improves fairness metrics, demonstrating improvements of at least 9.22% over baselines. This work highlights the importance of jointly optimizing architecture, training, and evaluation for fair, scalable spatial-temporal forecasting.

Abstract

Achieving both accurate and consistent predictive performance across spatial nodes is crucial for ensuring the validity and reliability of outcomes in fair spatial-temporal forecasting tasks. However, existing training methods treat heterogeneous nodes with a fully averaged perspective, resulting in inherently biased prediction targets. Balancing accuracy and consistency is particularly challenging due to the multi-objective nature of spatial-temporal forecasting. To address this issue, we propose a novel Heterogeneity-Informed Mixture-of-Experts (HiMoE) framework that delivers both uniform and precise spatial-temporal predictions. From a model architecture perspective, we design the Heterogeneity-Informed Graph Convolutional Network (HiGCN) to address trend heterogeneity, and we introduce the Node-wise Mixture-of-Experts (NMoE) module to handle cardinality heterogeneity across nodes. From an evaluation perspective, we propose STFairBench, a benchmark that handles fairness in spatial-temporal prediction from both training and evaluation stages. Extensive experiments on four real-world datasets demonstrate that HiMoE achieves state-of-the-art performance, outperforming the best baseline by at least 9.22% across all evaluation metrics.

Figures (7)

• Figure 1: In the PEMS04 dataset, the prediction performances of different nodes under identical Mean Absolute Error (MAE) exhibit substantial disparities, primarily driven by inherent variations in node-specific traffic volumes.
• Figure 2: The overall architecture of the HiMoE model. The solid boxes represent the model pipeline, while the dashed boxes illustrate how the Mixture of Experts operates and outline the structure of the expert models.
• Figure 3: Ablation Study on PeMS04 dataset.
• Figure 4: Comparison of baseline model performance with different loss functions: the left bar represents the model performance trained with MAE loss, while the right bar represents the model performance trained with the loss from STFairBench.
• Figure 5: Parameter sensitivity with respect to the loss function coefficient $\alpha$.