Constraint- and Score-Based Nonlinear Granger Causality Discovery with Kernels

TL;DR

This work addresses nonlinear time-series causal discovery by unifying two kernel-based constraint approaches under Kernel Principal Component Regression (KPCR) and introducing a Gaussian Process score-based method with Smooth Information Criterion (SIC) penalisation for improved causal identification. The KPCR framework subsumes kernel Granger causality (KGC) and large-scale nonlinear Granger causality (lsNGC) and enables a statistically principled F-test-based edge detection, with Nyström approximation enabling scalability. The GP_SIC model provides a competitive, fully GC-based score for both lagged and contemporaneous causal discovery, incorporating an ARD kernel and a differentiable sparsity penalty to select relevant features. The paper demonstrates through extensive numerical simulations and a real-world pH neutralisation plant application that these kernel-based methods offer strong performance, with GP_SIC particularly excelling in high-dimensional or highly autocorrelated settings. Overall, the results suggest practical, scalable tools for discovering nonlinear and contemporaneous causal structure in complex time-series systems, with clear avenues for future work on non-Gaussian noise and broader SCM assumptions.

Abstract

Kernel-based methods are used in the context of Granger Causality to enable the identification of nonlinear causal relationships between time series variables. In this paper, we show that two state of the art kernel-based Granger Causality (GC) approaches can be theoretically unified under the framework of Kernel Principal Component Regression (KPCR), and introduce a method based on this unification, demonstrating that this approach can improve causal identification. Additionally, we introduce a Gaussian Process score-based model with Smooth Information Criterion penalisation on the marginal likelihood, and demonstrate improved performance over existing state of the art time-series nonlinear causal discovery methods. Furthermore, we propose a contemporaneous causal identification algorithm, fully based on GC, using the proposed score-based $GP_{SIC}$ method, and compare its performance to a state of the art contemporaneous time series causal discovery algorithm.

Figures (10)

• Figure 1: Comparison of four GP-based Granger Causality Methods -- the $GP_{SIC}$ method, single model GP method without SIC cui_gaussian_2022, two-model likelihood difference method $GP \Delta_\ell$amblard_gaussian_2012, and the GP GLRT method zaremba_statistical_2022. The methods were compared across a selection of the simulated experiments.
• Figure 2: F1 score for five nonlinear causal discovery methods: $GP_{SIC}$, lsNGC, KGC, KPCR method, and PCMCI. 100 MC experiments were run for each simulated dataset for $n=250, 500$. The median is marked with a $\star$.
• Figure 3: $n=500$ -- For each pair of causal discovery methods $(C_1, C_2)$ (e.g., $C_1 = GP_{SIC}$ and $C_2 = PCMCI$), we report the posterior probability vector that $C_1 > C_2$, $C_1 \equiv C_2$ and $C_1 < C_2$ denoted respectively as $[p(C_1),\; p(rope),\; p(C_2)]$, obtained via the Bayesian Wilcoxon signed-rank test. The cloud of points in the figure corresponds to samples from this posterior distribution: each point is a probability vector $[p(C_1),\; p(rope),\; p(C_2)]$ which we plot in the probability simplex.
• Figure 4: Box plots of the F1 score for the contemporaneous $GP_{SIC}$ algorithm and $PCMCI+$. The F1 score is evaluated separately for the lagged adjacencies, the contemporaneous adjacencies, and the contemporaneous orientations. The plots on the left are linear Gaussian systems, and the plots on the right are nonlinear Gaussian systems.
• Figure 5: $n = 250$ -- For each pair of causal discovery methods $(C_1, C_2)$ (e.g., $C_1 = GP_{SIC}$ and $C_2 = PCMCI$), we report the posterior probability vector that $C_1 > C_2$, $C_1 \equiv C_2$ and $C_1 < C_2$ denoted respectively as $[p(C_1),\; p(rope),\; p(C_2)]$, obtained via the Bayesian Wilcoxon signed-rank test. The cloud of points in the figure corresponds to samples from this posterior distribution: each point is a probability vector $[p(C_1),\; p(rope),\; p(C_2)]$ which we plot in the probability simplex.

Theorems & Definitions (13)

• Proposition 1
• Proposition 2
• Remark 1
• Theorem 1
• Theorem 2
• proof
• proof
• Lemma 1
• proof
• Lemma 2