RPMixer: Shaking Up Time Series Forecasting with Random Projections for Large Spatial-Temporal Data

TL;DR

RPMixer tackles large-scale spatial-temporal forecasting without relying on input graphs by employing an all-MLP mixer augmented with fixed random projection layers and FFT-based frequency-domain processing. Viewing mixer blocks through an ensemble lens via identity-mapped residual connections, the method enhances diversity among block outputs and improves predictive accuracy on the LargeST benchmarks. Across SD, GBA, GLA, and CA, RPMixer outperforms graph-based and general forecasting baselines, with ablation analyses confirming the critical roles of pre-activation identity paths and random projections. The work demonstrates strong scalability (linear memory with node count) and points toward integrating random-projection concepts with time-series foundation models for future advances.

Abstract

Spatial-temporal forecasting systems play a crucial role in addressing numerous real-world challenges. In this paper, we investigate the potential of addressing spatial-temporal forecasting problems using general time series forecasting models, i.e., models that do not leverage the spatial relationships among the nodes. We propose a all-Multi-Layer Perceptron (all-MLP) time series forecasting architecture called RPMixer. The all-MLP architecture was chosen due to its recent success in time series forecasting benchmarks. Furthermore, our method capitalizes on the ensemble-like behavior of deep neural networks, where each individual block within the network behaves like a base learner in an ensemble model, particularly when identity mapping residual connections are incorporated. By integrating random projection layers into our model, we increase the diversity among the blocks' outputs, thereby improving the overall performance of the network. Extensive experiments conducted on the largest spatial-temporal forecasting benchmark datasets demonstrate that the proposed method outperforms alternative methods, including both spatial-temporal graph models and general forecasting models.

Figures (15)

• Figure 1: Each row consists of the intermediate and final outputs from a sampled node, while each column consists of the outputs from different mixer blocks within the model. The three samples are from node 238, 280, and 295 of the SD dataset (see Section \ref{['sec:experiment']} for details).
• Figure 2: The time series for three randomly selected nodes from the LargeST dataset spans a period of four weeks.
• Figure 3: The RPMixer architecture.
• Figure 4: The detailed design of the mixer block. The red line highlighted the identity mapping connection for a mixer block.
• Figure 5: The complex linear layer.

Theorems & Definitions (2)

• Definition 1
• Definition 2