Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse Graphical Linear Dynamical Systems

Emilie Chouzenoux, Victor Elvira

TL;DR

This work addresses the lack of joint static and dynamic graphical modeling within linear-Gaussian state-space models by introducing DGLASSO, a Bayesian MAP framework that sparsifies both the state-transition matrix $A$ (directed graph) and the state-noise precision $P=Q^{-1}$ (undirected Gaussian graphical model). The inference employs a block alternating proximal majorization-minimization algorithm with inner proximal solvers, backed by convergence guarantees via Kurdyka-Łojasiewicz theory. Extensive experiments on synthetic LG-SSMs and weather-inspired datasets demonstrate that DGLASSO yields superior edge-detection performance and predictive accuracy compared to EM-based, GLASSO-based, and Granger-causality baselines, while providing interpretable latent-graph structures. The approach enables simultaneous learning of two complementary graphical representations from a single observed time series and delivers full filtering/smoothing distributions for time-series inference, with public code to facilitate reproducibility and applicability to diverse domains.

Abstract

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm.

Sparse Graphical Linear Dynamical Systems

TL;DR

This work addresses the lack of joint static and dynamic graphical modeling within linear-Gaussian state-space models by introducing DGLASSO, a Bayesian MAP framework that sparsifies both the state-transition matrix (directed graph) and the state-noise precision (undirected Gaussian graphical model). The inference employs a block alternating proximal majorization-minimization algorithm with inner proximal solvers, backed by convergence guarantees via Kurdyka-Łojasiewicz theory. Extensive experiments on synthetic LG-SSMs and weather-inspired datasets demonstrate that DGLASSO yields superior edge-detection performance and predictive accuracy compared to EM-based, GLASSO-based, and Granger-causality baselines, while providing interpretable latent-graph structures. The approach enables simultaneous learning of two complementary graphical representations from a single observed time series and delivers full filtering/smoothing distributions for time-series inference, with public code to facilitate reproducibility and applicability to diverse domains.

Abstract

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm.
Paper Structure (48 sections, 3 theorems, 71 equations, 10 figures, 4 tables)

This paper contains 48 sections, 3 theorems, 71 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Graphical inference.
  3. The need for better graphical modeling in SSMs.
  4. Summary of our main contributions
  5. Preliminaries
  6. Notation
  7. Graphs
  8. Dynamical modeling
  9. Considered model
  10. Filtering and smoothing algorithms in linear dynamical systems
  11. Problem statement
  12. Proposed model and inference method
  13. Sparse dynamical graphical model
  14. State transition matrix
  15. State noise precision matrix
  16. ...and 33 more sections

Key Result

Theorem 1

The loss function can be expressed asNote that the l.h.s. in eq:lossexplicit does not depend on ${\mathbf x}_{0:K}$, so the r.h.s. is valid for any arbitrary value of ${\mathbf x}_{0:K}$ with non-zero probability under $p({\mathbf x}_{0:K})$, i.e., for all ${\mathbf x}_{0:K} \in \mathbb{R}^{(K+1)N_x Moreover, consider the outputs of Algorithms alg_kf and alg_rts for a given $\widetilde{{\mathbf A}

Figures (10)

  • Figure 1: Graphical model associated to \ref{['eq:stateExample']}. Matrix ${\mathbf A}$ (a), its binary support (b) and associated directed graph (c). The edges are defined as non-zero entries of ${\mathbf A}^\top$. Non-zero diagonal entries result in self-loops (here, in vertex $1$). The thickness of arrows is proportional to the absolute entries of ${\mathbf A}^\top$.
  • Figure 2: Matrix ${\mathbf P}$ (a), its binary support (b), and the associated undirected graph (c) with edge thickness proportional to the absolute entries of ${\mathbf P}$. Self-loops are removed, for readability purpose.
  • Figure 3: Summary representation of the DGLASSO graphical model, for the example graphs presented in Figs. \ref{['fig:adj_A']} and \ref{['fig:adj_P']}. Blue (oriented) edges represent Granger causality between state entries among consecutive time steps, encoded in matrix ${\mathbf A}$ (Fig. \ref{['fig:adj_A']}). Magenta edges represent static (i.e., instantaneous) relationships between the state entries, at every time step, due to correlated state noise described by matrix ${\mathbf P}$ (Fig. \ref{['fig:adj_P']}).
  • Figure 4: Evolution of RMSE, F1, cNMSE and loss scores on the estimation of ${\mathbf A}$ (left) and ${\mathbf P}$ (right) by DGLASSO, as a function of hyperparameters $(\lambda_A,\lambda_P)$, for dataset A (averaged on 10 runs). As a comparison, the averaged RMSE scores for $(\lambda_A,\lambda_P) = (0,0)$ (i.e., MLEM) on this example were $(0.077,0.106)$ for $({\mathbf A},{\mathbf P})$, respectively.
  • Figure 5: Box plots for quantitative metrics when running MLEM, and DGLASSO, on 50 randomly generated LG-SSM time series, for dataset A (top) and dataset D (bottom). F1 score is not reported for MLEM, as this method does not perform edge detection, resulting in a constant F1 score around $0.5$. DGLASSO outperforms MLEM with better (i.e., lower) RMSE scores for most runs and good F1 scores. Dataset D is more challenging in terms of inference, thus yielding to more spread results for both methods.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 1
  • Lemma 1
  • Theorem 2