Causal-INSIGHT: Probing Temporal Models to Extract Causal Structure

Abstract

Understanding directed temporal interactions in multivariate time series is essential for interpreting complex dynamical systems and the predictive models trained on them. We present Causal-INSIGHT, a model-agnostic, post-hoc interpretation framework for extracting model-implied (predictor-dependent), directed, time-lagged influence structure from trained temporal predictors. Rather than inferring causal structure at the level of the data-generating process, Causal-INSIGHT analyzes how a fixed, pre-trained predictor responds to systematic, intervention-inspired input clamping applied at inference time. From these responses, we construct directed temporal influence signals that reflect the dependencies the predictor relies on for prediction, and introduce Qbic, a sparsity-aware graph selection criterion that balances predictive fidelity and structural complexity without requiring ground-truth graph labels. Experiments across synthetic, simulated, and realistic benchmarks show that Causal-INSIGHT generalizes across diverse backbone architectures, maintains competitive structural accuracy, and yields significant improvements in temporal delay localization when applied to existing predictors.

Figures (5)

• Figure 1: Overview of Causal-INSIGHT. The method is organized into two stages: (left) the prediction problem, and (right) the causal graph construction problem. The analysis proceeds as follows: (A) Training of a temporal predictor $f_\theta$ under causal (no-leakage) masking constraints. (B) Model probing via input clamping at inference time with the predictor held fixed. (C) Construction of causal influence signals by comparing clamped and unclamped predictions. (D) Temporal reduction of influence signals to obtain a soft structural adjacency with associated peak lags. (E) Qbic scoring of candidate graphs using parent-based masking with a sparsity penalty. (F) Selection of the final directed causal graph by minimizing Qbic over sparsity levels.
• Figure 2: Qualitative comparison of predicted adjacency matrices on a representative fMRI dataset (Sim5). CausalFormer yields a highly conservative graph dominated by self-dependencies (diagonal), whereas Causal-INSIGHT recovers additional directed interactions while preserving these self-loops, resulting in higher structural accuracy (F1 $=0.80$). Misclassifications are outlined in red.
• Figure 3: Runtime scaling with the number of variables. Mean wall-clock runtime over three runs for CausalFormer and Causal-INSIGHT (CF) with fixed sequence length $T$. Shaded regions denote ±1 standard deviation across three runs. Non-monotonic behavior reflects early stopping in Qbic graph selection.
• Figure 4: Qbic and structural F1 trajectories as a function of graph sparsity for a representative fMRI dataset (Sim3). The Qbic minimum occurs at an edge count $\hat{m}$ that yields near-maximal F1, demonstrating that Qbic reliably selects high-quality graph structures without access to ground-truth labels.
• Figure 5: Impact of signal quality on F1 scores using original versus permuted causal signals on fMRI data. Performance degradation under signal corruption is minor for low-dimensional systems but substantial for higher-dimensional fMRI datasets.