Deep Gaussian Markov Random Fields for Graph-Structured Dynamical Systems

TL;DR

This work extends Deep Gaussian Markov Random Fields to spatiotemporal graph-structured systems (ST-DGMRF) by formulating graph-structured dynamics as a joint space-time GMRF defined via simple spatial and temporal layers. It enables efficient variational learning from limited data and exact posterior inference using conjugate gradients, with linear scaling in the number of time steps and state dimensions under $\mathcal{O}(KN d_{\text{spatial}} L_{\text{spatial}} + KN d_{\text{temporal}} L_{\text{temporal}})$. Empirical results on synthetic advection-diffusion and real-world air quality data show ST-DGMRF outperforms baselines that rely on simplified dependencies, providing more accurate state estimates and well-calibrated uncertainty, even with substantial missing data. The approach combines principled Bayesian inference with scalable deep-layer parameterizations, offering a data-efficient and tractable framework for high-dimensional spatiotemporal inference with partially unknown dynamics.

Abstract

Probabilistic inference in high-dimensional state-space models is computationally challenging. For many spatiotemporal systems, however, prior knowledge about the dependency structure of state variables is available. We leverage this structure to develop a computationally efficient approach to state estimation and learning in graph-structured state-space models with (partially) unknown dynamics and limited historical data. Building on recent methods that combine ideas from deep learning with principled inference in Gaussian Markov random fields (GMRF), we reformulate graph-structured state-space models as Deep GMRFs defined by simple spatial and temporal graph layers. This results in a flexible spatiotemporal prior that can be learned efficiently from a single time sequence via variational inference. Under linear Gaussian assumptions, we retain a closed-form posterior, which can be sampled efficiently using the conjugate gradient method, scaling favourably compared to classical Kalman filter based approaches

Figures (11)

• Figure 1: ST-DGMRF overview. We reconstruct the latent states of a graph-structured dynamical system from partial and noisy observations (orange arrow with question mark). Temporal and spatial layers transform the state $\mathbf{x}$ to a standard Gaussian, which implicitly defines a space-time GMRF prior.
• Figure 2: Left: snapshot of the advection-diffusion data at time $k$=8, and reconstructions by the time-independent DGMRF and our ST-DGMRF with advection-diffusion $\mathbf{F}_k$. Center: corresponding time series for a single pixel (marked on the left). Shaded areas represent posterior mean $\pm$ std of a single run. Right: $\text{RMSE}_{\mu}$ as a function of the mask width (mean $\pm$ std over 5 runs).
• Figure 3: Stencil comparison. Center: absolute error between true and learned transition weights for the neural network based ST-DGMRF. Right: Pearson correlation for increasing temporal depth.
• Figure 4: ST-DGMRF posterior estimates (neural network$\mathbf{F}_k$ and $p=2$). Left: time series of log-transformed and normalized PM2.5 levels for a masked sensor. Light red areas represent posterior mean $\pm$ std of a single run. Unobserved time points are marked with gray bars. Right: Marginal std for all sensors at $k=100$. Masked out sensors (gray box) feature higher uncertainties.
• Figure 5: Advection-diffusion dataset with ground truth system states (bottom) and corresponding observations using masks of width $w=9$ (top).