Towards Expressive Spectral-Temporal Graph Neural Networks for Time Series Forecasting

TL;DR

This work develops a theoretical framework for expressive spectral-temporal graph neural networks (SPTGNNs) applied to time series forecasting. It proves that linear SPTGNNs are universal under mild conditions and that their expressive power is bounded by a temporal WL test, offering a principled blueprint for designing spectral spatial and temporal modules. The authors introduce TGGC, a simple yet powerful linear instantiation based on Gegenbauer spectral filters and frequency-domain temporal modeling, which outperforms many baselines and demonstrates the ability to learn differently signed inter-series relationships. The study provides both rigorous theory and practical mechanisms to build provably expressive GNN-based time-series models, with TGGC serving as a concrete validation of the framework and a stepping stone toward broader SPTGNN families.

Abstract

Time series forecasting has remained a focal point due to its vital applications in sectors such as energy management and transportation planning. Spectral-temporal graph neural network is a promising abstraction underlying most time series forecasting models that are based on graph neural networks (GNNs). However, more is needed to know about the underpinnings of this branch of methods. In this paper, we establish a theoretical framework that unravels the expressive power of spectral-temporal GNNs. Our results show that linear spectral-temporal GNNs are universal under mild assumptions, and their expressive power is bounded by our extended first-order Weisfeiler-Leman algorithm on discrete-time dynamic graphs. To make our findings useful in practice on valid instantiations, we discuss related constraints in detail and outline a theoretical blueprint for designing spatial and temporal modules in spectral domains. Building on these insights and to demonstrate how powerful spectral-temporal GNNs are based on our framework, we propose a simple instantiation named Temporal Graph Gegenbauer Convolution (TGGC), which significantly outperforms most existing models with only linear components and shows better model efficiency. Our findings pave the way for devising a broader array of provably expressive GNN-based models for time series.

Figures (14)

• Figure 1: Differently signed spatial relations between time series in a real-world traffic (PeMS07) dataset. Left: Visualization of four randomly selected traffic sensor readings. Right: Spatial relations between time series may be different in two windows (e.g., A and B are positively and negatively correlated in windows 1 and 2, respectively), where the (weighted) edges between the sensors normally indicate their correlation strengths but not reflect the signs. Unlike most spatio-temporal GNNs that aggregate the neighborhood information without considering correlation signs, spectral-temporal GNNs go beyond low-pass filtering by learning to aggregate or differentiate such information.
• Figure 2: An overview of the theoretical results in this work.
• Figure 3: The general formulation of SPTGNNs with $M$ building blocks to predict future values $\Hat{\mathbf{Y}}$ based on historical observations $\mathbf{X}$. We denote $g_\theta(\cdot)$ and $\mathcal{T}_\phi(\cdot)$ as graph and temporal spectral filters. $\mathcal{F}(\cdot)$ and $\mathcal{F}^{-1}(\cdot)$ are forward and inverse space projections.
• Figure 4: Multidimensional and multivariate predictions. Left: Multidimensional predictions within a snapshot require individual filtration for each output dimension to preserve different information.Right: Individual filter is needed to model each single-dimensional time series.
• Figure 5: Two examples of temporal 1-WL test on non-attributive discrete-time dynamic graphs. The left test fails to distinguish non-isomorphic nodes at $t_1$, e.g., A and C, while the right example demonstrates a successful test.

Theorems & Definitions (18)

• Definition 1: Graph Convolution
• Definition 2
• Proposition 1
• Theorem 1
• Definition 3: Temporal 1-WL test
• Theorem 2
• Proposition 2
• Proposition 3
• Theorem 3
• Lemma 1