A VAE-based Framework for Learning Multi-Level Neural Granger-Causal Connectivity

TL;DR

This paper introduces a Variational Autoencoder (VAE) based framework that jointly learns Granger-causal relationships amongst components in a collection of related-yet-heterogeneous dynamical systems, and handles the aforementioned task in a principled way.

Abstract

Granger causality has been widely used in various application domains to capture lead-lag relationships amongst the components of complex dynamical systems, and the focus in extant literature has been on a single dynamical system. In certain applications in macroeconomics and neuroscience, one has access to data from a collection of related such systems, wherein the modeling task of interest is to extract the shared common structure that is embedded across them, as well as to identify the idiosyncrasies within individual ones. This paper introduces a Variational Autoencoder (VAE) based framework that jointly learns Granger-causal relationships amongst components in a collection of related-yet-heterogeneous dynamical systems, and handles the aforementioned task in a principled way. The performance of the proposed framework is evaluated on several synthetic data settings and benchmarked against existing approaches designed for individual system learning. The method is further illustrated on a real dataset involving time series data from a neurophysiological experiment and produces interpretable results.

Figures (25)

• Figure 1: Diagram for the postulated top-down generative process.
• Figure 2: Diagram for the end-to-end encoding-decoding procedure. Solid paths with arrows denote modeling the corresponding distributions during the encoding/decoding process; dashed paths with arrows correspond to information merging based on (weighted) conjugacy adjustment. Quantities obtained after each step are given inside the circles/rectangles. $\{\mathbb{x}^{[m]}\}$ is short for the collection $\{\mathbb{x}^{[m]}\}_{m=1}^M$; $\{\mathbf{z}^{[m]}\}$ is analogously defined.
• Figure 3: True (shaded panel on the left) and estimated (non-shaded panel on the right) Granger-causal connections using the proposed framework with node-centric decoder (Multi-node); from left to right: $\bar{\mathbf{z}}$, $\mathbf{z}^{[1]}$ and $\mathbf{z}^{[2]}$ and their estimated counterparts.Nonzero entries in $\mathbf{z}^{[1]},\mathbf{z}^{[2]}$ (and $\widehat{\mathbf{z}}^{[1]},\widehat{\mathbf{z}}^{[2]}$, resp.) that overlap with those in $\bar{\mathbf{z}}$ ($\widehat{\bar{\mathbf{z}}}$) have been grayed-out so that the idiosyncratic ones stand out.
• Figure 4: Multi-node results: estimated common Granger-causal connections for EO (left panel) and EC (right panel) after normalization and subsequent thresholding at 0.15. Red edges correspond to positive connections and blue edges correspond to negative ones; the transparency of the edges is proportional to the strength of the connection. Larger node sizes correspond to higher in-degree (incoming connectivity), and the top 6 nodes are colored in gray.
• Figure 5: GVAR results: estimated common Granger-causal connections for EO (left panel) and EC (right panel) after normalization and subsequent thresholding at 0.05. Red edges correspond to positive connections and blue edges correspond to negative ones; the transparency of the edges is proportional to the strength of the connection. Larger node sizes correspond to higher in-degree (incoming connectivity), and the top 6 nodes are colored in gray.

Theorems & Definitions (6)

• Remark 1: On the proposed formulation
• Remark 2: On the role of $\omega$
• Remark 3
• Remark 4: On the robustness with respect to sample size
• Remark 5
• Remark 6: On the estimated Granger-causal graph