Lightweight Spatio-Temporal Attention Network with Graph Embedding and Rotational Position Encoding for Traffic Forecasting
Xiao Wang, Shun-Ren Yang
TL;DR
Traffic forecasting is challenged by complex spatio-temporal dependencies and limitations of short-range spatial modeling. The authors introduce LSTAN-GERPE, a lightweight spatio-temporal attention network that combines a data embedding block, a stack of $K=5$ spatio-temporal attention pairs, Rotary Position Encoding with tunable frequencies $\Theta^{(t)}$, and eigenvector-based graph features to enrich spatial representations. Key contributions include the parallel spatial/temporal STA design, RoPE with frequency control, and Laplacian eigenvector embedding to incorporate map topology, all without heavy feature engineering. Empirical results on PeMS04 and PeMS08 show strong performance against a wide set of baselines, with ablations confirming the usefulness of RoPE, graph embeddings, and dual attention streams. The approach offers a scalable, accurate option for real-world ITS deployments and motivates future extensions to broader spatio-temporal data.
Abstract
Traffic forecasting is a key task in the field of Intelligent Transportation Systems. Recent research on traffic forecasting has mainly focused on combining graph neural networks (GNNs) with other models. However, GNNs only consider short-range spatial information. In this study, we present a novel model termed LSTAN-GERPE (Lightweight Spatio-Temporal Attention Network with Graph Embedding and Rotational Position Encoding). This model leverages both Temporal and Spatial Attention mechanisms to effectively capture long-range traffic dynamics. Additionally, the optimal frequency for rotational position encoding is determined through a grid search approach in both the spatial and temporal attention mechanisms. This systematic optimization enables the model to effectively capture complex traffic patterns. The model also enhances feature representation by incorporating geographical location maps into the spatio-temporal embeddings. Without extensive feature engineering, the proposed method in this paper achieves advanced accuracy on the real-world traffic forecasting datasets PeMS04 and PeMS08.