FaST: Efficient and Effective Long-Horizon Forecasting for Large-Scale Spatial-Temporal Graphs via Mixture-of-Experts

TL;DR

FaST tackles the challenge of long-horizon forecasting on large-scale spatial-temporal graphs by introducing a heterogeneity-aware Mixture-of-Experts (MoE) framework that compresses temporal history and a lightweight, adaptive graph attention mechanism. The key innovations—Temporal Compression Input with MoE, Heterogeneity-Aware MoE with a load-balancing router, and Adaptive Graph Agent Attention with a GLU-based parallel MoE—achieve linear time and memory scaling in both the number of nodes and the forecasting horizon while preserving rich spatiotemporal dependencies. Empirical results on the LargeST benchmark show FaST consistently surpasses state-of-the-art baselines in accuracy and efficiency for horizon lengths up to 672 steps, with substantial memory and compute savings. These advances enable practical one-week-ahead forecasting over thousands of sensors, offering impactful implications for urban sensing and smart infrastructure management.

Abstract

Spatial-Temporal Graph (STG) forecasting on large-scale networks has garnered significant attention. However, existing models predominantly focus on short-horizon predictions and suffer from notorious computational costs and memory consumption when scaling to long-horizon predictions and large graphs. Targeting the above challenges, we present FaST, an effective and efficient framework based on heterogeneity-aware Mixture-of-Experts (MoEs) for long-horizon and large-scale STG forecasting, which unlocks one-week-ahead (672 steps at a 15-minute granularity) prediction with thousands of nodes. FaST is underpinned by two key innovations. First, an adaptive graph agent attention mechanism is proposed to alleviate the computational burden inherent in conventional graph convolution and self-attention modules when applied to large-scale graphs. Second, we propose a new parallel MoE module that replaces traditional feed-forward networks with Gated Linear Units (GLUs), enabling an efficient and scalable parallel structure. Extensive experiments on real-world datasets demonstrate that FaST not only delivers superior long-horizon predictive accuracy but also achieves remarkable computational efficiency compared to state-of-the-art baselines. Our source code is available at: https://github.com/yijizhao/FaST.

Figures (7)

• Figure 1: Efficiency--effectiveness comparison on large-scale STG benchmark (CA dataset; 8,600 nodes; 672-step prediction horizon). Smaller bubble means faster inference speed. The proposed FaST achieves the best performance and speed (both in training and inference).
• Figure 2: Architecture of FaST. The middle part illustrates the workflow: the input sequence ${\bf X}_t$ is first embedded and fed into $L$ stacked backbone blocks, and the resulting representations are concatenated and passed to an MLP predictor to generate $\hat{{\bf Y}}_t$.
• Figure 3: Long-horizon forecasting performance comparison. "96=>672" denotes 672 steps ahead forecasting based on the past 96 time steps. "T" refers to temporal-centric methods, while "ST" denotes spatial-temporal-centric methods. FaST achieved the best performance on 16 prediction tasks.
• Figure 4: Ablation study of FaST components.
• Figure 5: Visualization of hyperparameter sensitivity.