SDA-GRIN for Adaptive Spatial-Temporal Multivariate Time Series Imputation

TL;DR

Missing data in multivariate time series is addressed by treating the data as a sequence of temporal graphs with $N$ nodes and window length $T$. SDA-GRIN uses scaled dot-product Multi-Head Attention to compute time-varying adjacencies $A_t^* = \hat{A}_t^{pooled} \odot (A > 0)$ and a MPGRU-based imputation backbone to perform imputation. Evaluations on four real-world datasets show improvements over baselines, with MSE reductions of up to $9.51\%$ on AQI, $9.40\%$ on AQI-36, and $1.94\%$ on PEMS-BAY. The approach is particularly effective when inter-variable spatial relationships exhibit high temporal variability and suggests broad applicability to IoT sensing networks where spatial relations drift over time.

Abstract

In various applications, the multivariate time series often suffers from missing data. This issue can significantly disrupt systems that rely on the data. Spatial and temporal dependencies can be leveraged to impute the missing samples. Existing imputation methods often ignore dynamic changes in spatial dependencies. We propose a Spatial Dynamic Aware Graph Recurrent Imputation Network (SDA-GRIN) which is capable of capturing dynamic changes in spatial dependencies.SDA-GRIN leverages a multi-head attention mechanism to adapt graph structures with time. SDA-GRIN models multivariate time series as a sequence of temporal graphs and uses a recurrent message-passing architecture for imputation. We evaluate SDA-GRIN on four real-world datasets: SDA-GRIN improves MSE by 9.51% for the AQI and 9.40% for AQI-36. On the PEMS-BAY dataset, it achieves a 1.94% improvement in MSE. Detailed ablation study demonstrates the effect of window sizes and missing data on the performance of the method. Project page:https://ameskandari.github.io/sda-grin/

Figures (4)

• Figure 1: Overview of SDA-GRIN. The multivariate time series (MTS) is chunked into windows. (a) MHA calculates attention among variables within each window, generating attention weights. These are pooled and used to adapt the static graph with time, creating $A^*_t$. (b) Later, in the unidirectional setting, the model processes the adapted graph ($A^*_t$), masked input ($X_t$), and previous context ($H_{t-1}$), outputting first-stage, second-stage imputed samples ($\hat{X}_t^{(1)}$, $\hat{X}_t^{(2)}$), and updated context ($H_t$).
• Figure 2: Pair-wise Relative Mean Squared Error (RMSE) among all variables at each step, shown for AQI, AQI-36, PEMS-BAY, and METR-LA datasets. We use 128 as the window size for the validation part of the datasets. The line represents the mean, while the margin indicates the standard deviation of pair-wise RMSE of all variables. AQI and AQI-36 datasets show greater values and variation within a window and across different windows compared to METR-LA and PEMS-BAY.
• Figure 3: Performance of SDA-GRIN across different missing rates, ranging from 10% to 90% in 10% increments. Performance declines at higher missing rates due to the reduced availability of samples within each variable for MHA to attend to.
• Figure 4: Comparison of the static adjacency graph with three random dynamic graphs for AQI36.