Higher-order Spatio-temporal Physics-incorporated Graph Neural Network for Multivariate Time Series Imputation

TL;DR

This paper addresses missing values in multivariate time series on non-Euclidean sensor graphs. It introduces HSPGNN, which integrates a dynamic Laplacian $L_t$ learned via spatial attention and a generic inhomogeneous PDE that combines high-order time derivatives $\sum_m \lambda_m \partial_t^m u$ with spatial derivatives $\sum_k \Theta_k \partial_x^k \partial_y^k u$, enabling adaptive multi-order physics-informed modeling. A coarse imputation via MLP, a physics-incorporated high-order GNN layer, a future-state predictor using LSTM and Temporal Attention, and a Normalizing Flows–based explainability module for node-missing impact form the core of the method. Experiments on four benchmarks show robust imputation under various missing patterns and reveal dynamic graph structures and graph-like optical-flow that provide interpretability beyond standard data-driven models.

Abstract

Exploring the missing values is an essential but challenging issue due to the complex latent spatio-temporal correlation and dynamic nature of time series. Owing to the outstanding performance in dealing with structure learning potentials, Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) are often used to capture such complex spatio-temporal features in multivariate time series. However, these data-driven models often fail to capture the essential spatio-temporal relationships when significant signal corruption occurs. Additionally, calculating the high-order neighbor nodes in these models is of high computational complexity. To address these problems, we propose a novel higher-order spatio-temporal physics-incorporated GNN (HSPGNN). Firstly, the dynamic Laplacian matrix can be obtained by the spatial attention mechanism. Then, the generic inhomogeneous partial differential equation (PDE) of physical dynamic systems is used to construct the dynamic higher-order spatio-temporal GNN to obtain the missing time series values. Moreover, we estimate the missing impact by Normalizing Flows (NF) to evaluate the importance of each node in the graph for better explainability. Experimental results on four benchmark datasets demonstrate the effectiveness of HSPGNN and the superior performance when combining various order neighbor nodes. Also, graph-like optical flow, dynamic graphs, and missing impact can be obtained naturally by HSPGNN, which provides better dynamic analysis and explanation than traditional data-driven models. Our code is available at https://github.com/gorgen2020/HSPGNN.

Figures (7)

• Figure 1: The framework of HSPGNN. Firstly, coarse imputation value $\mathbf{\Bar{P}}$ can be estimated by MLP. Then, on the left, the latent dynamic spatial graphs $\boldsymbol{\mathcal{G}_t}$ structure is learned from the estimated value, providing the dynamic Laplacian matrix. On the right, a novel physics-incorporated GNN layer is devised, which can combine multi-order spatial and temporal neighbor signals into consideration. As for the physics-incorporated GNN layer, physical dynamic system inhomogeneous PDE is applied to capture the spatio-temporal correlation and perform more accurate imputed values $\hat{\mathbf{P}}_{t-M:t}$. Thus, the imputed values are utilized to predict the future value $\hat{\mathbf{X}}_{t:t+M} \odot (1-\mathbf{M}_{t:t+M})$ by LSTM and temporal attention since the ground truth of the missing values are not available in the training stage.
• Figure 2: MAE of different models at different missing rates on PeMS-BAY and Electricity datasets.
• Figure 3: The graph-like optical flow at different month and weeks on AQI and PeMS-BAY datasets.
• Figure 4: Dynamic graphs at different time steps on Electricity dataset.
• Figure 5: The missing impact and estimated $p(\mathbf{P},\mathbf{Y}|\mathbf{Z})$ distribution on PeMS-BAY and Electricity datasets .

Theorems & Definitions (4)

• Theorem 3.1
• proof
• Theorem 3.2
• proof