Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Flow based approach for Dynamic Temporal Causal models with non-Gaussian or Heteroscedastic Noises

Abdellah Rahmani, Pascal Frossard

TL;DR

FANTOM addresses causal discovery from multi-regime time series with non-stationarity and complex noise by jointly inferring the number of regimes, their boundaries, and regime-specific DAGs. It introduces a Bayesian EM framework that uses temporal graph neural networks and conditional normalizing flows to model non-Gaussian and heteroscedastic noise, while a principled initialization and regime-learning scheme progressively refines regime indices. The authors prove identifiability for stationary and non-stationary settings and validate the approach with extensive synthetic and real-world experiments, where FANTOM outperforms existing methods in both structure learning and regime detection. The work advances causal discovery in non-stationary environments, enabling regime-aware causal analysis with potential impact in finance, climate science, and neuroscience.

Abstract

Understanding causal relationships in multivariate time series is crucial in many scenarios, such as those dealing with financial or neurological data. Many such time series exhibit multiple regimes, i.e., consecutive temporal segments with a priori unknown boundaries, with each regime having its own causal structure. Inferring causal dependencies and regime shifts is critical for analyzing the underlying processes. However, causal structure learning in this setting is challenging due to (1) non-stationarity, i.e., each regime can have its own causal graph and mixing function, and (2) complex noise distributions, which may be nonGaussian or heteroscedastic. Existing causal discovery approaches cannot address these challenges, since generally assume stationarity or Gaussian noise with constant variance. Hence, we introduce FANTOM, a unified framework for causal discovery that handles non-stationary processes along with non-Gaussian and heteroscedastic noises. FANTOM simultaneously infers the number of regimes and their corresponding indices and learns each regime's Directed Acyclic Graph. It uses a Bayesian Expectation Maximization algorithm that maximizes the evidence lower bound of the data log-likelihood. On the theoretical side, we prove, under mild assumptions, that temporal heteroscedastic causal models, introduced in FANTOM's formulation, are identifiable in both stationary and non-stationary settings. In addition, extensive experiments on synthetic and real data show that FANTOM outperforms existing methods.

Flow based approach for Dynamic Temporal Causal models with non-Gaussian or Heteroscedastic Noises

TL;DR

FANTOM addresses causal discovery from multi-regime time series with non-stationarity and complex noise by jointly inferring the number of regimes, their boundaries, and regime-specific DAGs. It introduces a Bayesian EM framework that uses temporal graph neural networks and conditional normalizing flows to model non-Gaussian and heteroscedastic noise, while a principled initialization and regime-learning scheme progressively refines regime indices. The authors prove identifiability for stationary and non-stationary settings and validate the approach with extensive synthetic and real-world experiments, where FANTOM outperforms existing methods in both structure learning and regime detection. The work advances causal discovery in non-stationary environments, enabling regime-aware causal analysis with potential impact in finance, climate science, and neuroscience.

Abstract

Understanding causal relationships in multivariate time series is crucial in many scenarios, such as those dealing with financial or neurological data. Many such time series exhibit multiple regimes, i.e., consecutive temporal segments with a priori unknown boundaries, with each regime having its own causal structure. Inferring causal dependencies and regime shifts is critical for analyzing the underlying processes. However, causal structure learning in this setting is challenging due to (1) non-stationarity, i.e., each regime can have its own causal graph and mixing function, and (2) complex noise distributions, which may be nonGaussian or heteroscedastic. Existing causal discovery approaches cannot address these challenges, since generally assume stationarity or Gaussian noise with constant variance. Hence, we introduce FANTOM, a unified framework for causal discovery that handles non-stationary processes along with non-Gaussian and heteroscedastic noises. FANTOM simultaneously infers the number of regimes and their corresponding indices and learns each regime's Directed Acyclic Graph. It uses a Bayesian Expectation Maximization algorithm that maximizes the evidence lower bound of the data log-likelihood. On the theoretical side, we prove, under mild assumptions, that temporal heteroscedastic causal models, introduced in FANTOM's formulation, are identifiable in both stationary and non-stationary settings. In addition, extensive experiments on synthetic and real data show that FANTOM outperforms existing methods.

Paper Structure

This paper contains 45 sections, 8 theorems, 49 equations, 14 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related work.
  3. Problem formulation
  4. Flow based approach for Dynamic Temporal Causal models
  5. ELBO formulation
  6. Model parametrization
  7. BEM algorithm
  8. M step: graph learning
  9. E step: Regime learning
  10. Identifiability results
  11. Experiments
  12. Synthetic data
  13. Real world data
  14. Conclusion
  15. Detailed related works
  16. ...and 30 more sections

Key Result

Proposition 3.1

Let $(\boldsymbol{x}_t)_{t \in \mathcal{T}}$ be a MTS composed of multiple regimes and following the SEM described in Eq.(eq_multi_reg_hetero). The data likelihood admits the following evidence lower bound (ELBO): where $\forall t \in \mathcal{T}: z_t \in [|1:N_w|]$ are the discrete latent variables and $N_w$ is the number of regimes.

Figures (14)

  • Figure 1: Illustration of FANTOM processing a MTS with two ground truth regimes ($K=2$). The algorithm recovers the regime indices $\mathcal{I}^{*}_1$ and $\mathcal{I}^{*}_2$ and learns a temporal causal graph for each regime (dashed edges represent time-lagged links; solid arrows indicate instantaneous links). In the E-step, posterior probabilities $p\bigl(z_t = r \mid \boldsymbol{x}_t, \boldsymbol{x}_{<t}\bigr)$ are estimated, where $z_t = r$ means $\boldsymbol{x}_t$ belongs to regime $r$. The M-step then infers causal graphs within each regime. Here, $N_w$ is the number of regimes that converges to $K=2$.
  • Figure 2: Temporal graph neural network (TGNN) used by FANTOM.
  • Figure 3: Illustration of $\pi_t\left(\boldsymbol{\omega}^r\right)$ after Fantom's first iteration with equal windows of 1500 samples for an MTS of 4500 samples with two ground-truth regimes: $\mathcal{I}^{*}_1 = [|0:1999|]$ and $\mathcal{I}^{*}_2 = [|2000:4500|]$.
  • Figure 4: Graphical model of FANTOM. Observed variables ($\boldsymbol{x}_t$) are in gray, while latent variables ($z_t$) and parameters ($\Theta$) are in white. Blue edges represent parameter-variable interactions.
  • Figure 5: Illustration of $\pi_t\left(\boldsymbol{\omega}^r\right)$ after Fantom's first iteration with equal windows of 1500 samples for an MTS of 4500 samples with two ground-truth regimes: $\mathcal{I}^{*}_1 = [|0:1999|]$ and $\mathcal{I}^{*}_2 = [|2000:4500|]$.
  • ...and 9 more figures

Theorems & Definitions (12)

  • Definition 2.1: Temporal Causal Graph rahmanicausal
  • Proposition 3.1
  • Definition 4.1
  • Theorem 4.2: Identifiability of Temporal Heteroscedastic Gaussian noise model (THGNM)
  • Theorem 4.3: Identifiability of the mixture of identifiable temporal causal models
  • Definition H.1: Causal Stationarity, runge2018causal
  • Definition H.3
  • Lemma H.7
  • Theorem H.8: Identifiability of Temporal Non Gaussian noise model (TNGNM)
  • Lemma H.9
  • ...and 2 more