Jacobian Regularizer-based Neural Granger Causality

TL;DR

This work addresses the inefficiency and limited scope of existing neural Granger causality methods, which typically require training multiple models or rely on sparsity constraints on network weights. It introduces Jacobian Regularizer-based Neural Granger Causality (JRNGC), a unified framework that trains a single multivariate forecasting model augmented with an input-output Jacobian regularizer $\mathcal{L}_{Jac}$ to learn both summary and full-time Granger causality. JRNGC defines and regularizes the Jacobian matrix $J(\boldsymbol{x})=[\partial f_i/\partial x_k^{t-\alpha}]$, using either $\|J(\boldsymbol{x})\|_1$ or a random-projection approximated $\|J(\boldsymbol{x})\|_{\mathrm{F}}^2$ to enforce causality constraints and enable scalable inference; post-hoc analysis uses $J_{i,j,\alpha}=\partial x_j / \partial x_i^{(t-\alpha)}$ to construct lagged causal graphs. Empirically, JRNGC-L1 and JRNGC-F achieve competitive or superior AUROC and AUPRC across VAR, Lorenz-96, fMRI, DREAM-3, and CausalTime benchmarks, with lower model complexity and the ability to recover full-time causality. The approach is architecture-agnostic and supports alternative predictors (e.g., LSTMs), and the authors provide code to promote reproducibility and practical adoption in causal discovery for high-dimensional time series.

Abstract

With the advancement of neural networks, diverse methods for neural Granger causality have emerged, which demonstrate proficiency in handling complex data, and nonlinear relationships. However, the existing framework of neural Granger causality has several limitations. It requires the construction of separate predictive models for each target variable, and the relationship depends on the sparsity on the weights of the first layer, resulting in challenges in effectively modeling complex relationships between variables as well as unsatisfied estimation accuracy of Granger causality. Moreover, most of them cannot grasp full-time Granger causality. To address these drawbacks, we propose a Jacobian Regularizer-based Neural Granger Causality (JRNGC) approach, a straightforward yet highly effective method for learning multivariate summary Granger causality and full-time Granger causality by constructing a single model for all target variables. Specifically, our method eliminates the sparsity constraints of weights by leveraging an input-output Jacobian matrix regularizer, which can be subsequently represented as the weighted causal matrix in the post-hoc analysis. Extensive experiments show that our proposed approach achieves competitive performance with the state-of-the-art methods for learning summary Granger causality and full-time Granger causality while maintaining lower model complexity and high scalability.

Figures (7)

• Figure 1: Motivation and our proposed neural Granger causality method. To learn the true Granger causality, we need to estimate the importance of a variable in helping forecast another variable. For example, let's examine a simple scenario involving the summary causal relationship: $X_4\leftarrow X_2 \leftarrow X_1\rightarrow X_3\rightarrow X_4$. To comprehend this relationship, current neural Granger causality methods need to construct and train the same number of models as the dimensions of the variables to disentangle the importance of each variable and obtain Granger causality by incorporating sparsity penalties on the first layer of each model, as illustrated in figures a)-d). However, sparse first-layer network parameters will result in challenges in effectively modeling complex relationships between variables as well as unsatisfied estimation accuracy of Granger causality. In addition, it will get wrong Granger causality if we use a multivariate time series forecasting model because of the existence of a shared hidden layer, as exemplified in figure e). Instead, our method only needs to build and train a single multivariate time series forecasting model by introducing input-output Jacobian regularizer $\mathcal{L}_{J a c}$. Note that the numbers I, II, III, and IV mean the same model architecture with independent training for different variables and we used two-layer Perceptron for the convenience of illustration.
• Figure 2: The framework of the Residual MLP layer in this work. FC represents the fully connected layer.
• Figure 3: Performance comparisons on a 100-dimensional VAR dataset: AUROC, AUPRC, and the number of tunable parameters.
• Figure 4: Typical causal graphs for time series. The full-time causal graph entails including all intricate causal links and interactions among variables. In contrast, the summary causal graph streamlines this complexity, focusing on the most significant causal connections.
• Figure 5: Estimated summary Causal graph Results on VAR (10,3,5) dataset. 95% confidence interval shown.