Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data

TL;DR

This work addresses the identifiability gap in causal discovery under i.i.d. data by exploiting exchangeable, grouped data. It introduces causal de Finetti theorems that represent exchangeable processes as ICM-generative models with independent latent mechanisms, and proves an identifiability theorem showing unique causal graphs are recoverable when data are faithful to ICM(\mathcal{G}). It then provides a practical Causal-de-Finetti algorithm for learning from multi-environment data, with experiments showing superior performance and robustness to changing functional forms. The results offer a theoretical and algorithmic framework for disentangled causal structure identification in grouped data, with public code to enable replication and application.

Abstract

Constraint-based causal discovery methods leverage conditional independence tests to infer causal relationships in a wide variety of applications. Just as the majority of machine learning methods, existing work focuses on studying $\textit{independent and identically distributed}$ data. However, it is known that even with infinite i.i.d.$\ $ data, constraint-based methods can only identify causal structures up to broad Markov equivalence classes, posing a fundamental limitation for causal discovery. In this work, we observe that exchangeable data contains richer conditional independence structure than i.i.d.$\ $ data, and show how the richer structure can be leveraged for causal discovery. We first present causal de Finetti theorems, which state that exchangeable distributions with certain non-trivial conditional independences can always be represented as $\textit{independent causal mechanism (ICM)}$ generative processes. We then present our main identifiability theorem, which shows that given data from an ICM generative process, its unique causal structure can be identified through performing conditional independence tests. We finally develop a causal discovery algorithm and demonstrate its applicability to inferring causal relationships from multi-environment data. Our code and models are publicly available at: https://github.com/syguo96/Causal-de-Finetti

Figures (5)

• Figure 1: (a) is an illustration showing how i.i.d. data and certain exchangeable data differ in identifying the correct causal structure for a bivariate model. Each quadrant represents a causal structure, ${X \perp\!\!\!\!\perp Y}$, ${X \rightarrow Y}$, ${X \leftarrow Y}$. The inner circle represents i.i.d. regime and the outer circle represents certain exchangeable regime. Under i.i.d. data, one can only identify ${X \perp\!\!\!\!\perp Y}$, whereas certain exchangeable data (i.e., ICM-generative processes) enables one to identify unique causal structures. (b) illustrates a differentiating factor between de Finetti and causal de Finetti's representation on exchangeable data. Causal de Finetti disentangles the latents and substantiates causal mechanisms are independent in the sense latent parameters governing each mechanisms are statistically independent.
• Figure 2: An illustration demonstrates different conditional independence relationships contained in i.i.d. process and ICM-generative process. (a): A causal graph generated under an i.i.d. process; (b): A causal graph generated under ICM-generative process; Unrolling the inner plate notation from (b), we visualize the process with two samples. Causal graphs $X \rightarrow Y$ and $Y \rightarrow X$ generated under an i.i.d. process share the same conditional independences $\{\emptyset\}$ and are thus observationally equivalent. (c) and (d) show the corresponding graphs under ICM-generative processes. (c) has $X_1 \perp\!\!\!\!\perp Y_2 \mid X_2$ which does not hold in (d) and (d) has $X_1 \perp\!\!\!\!\perp Y_2 \mid Y_1$ which does not hold in (c). One can thus differentiate the bivariate causal direction in ICM-generative processes.
• Figure 3: Our method’s (“Causal-de-Finetti”) performance in identifying the correct underlying DAG, compared to the “CD-NOD”, “FCI”, “GES”, “NOTEARS”, "DirectLinGAM", "PC", "Random" baseline in bivariate and multivariate settings. Shown are the mean and $95\%$ confidence interval of the standard error of the mean for each method aggregated over $100$ experiments. "Causal-de-Finetti" identifies unique causal structures and is robust against changing functions across environments.
• Figure :
• Figure :

Theorems & Definitions (53)

• Definition 1: Exchangeable sequence
• Theorem 1: De Finetti's representation theorem deFinetti1931
• Definition 2: Exchangeable pairs
• Theorem 2: Causal de Finetti -- bivariate
• Definition 3: Exchangeable arrays
• Theorem 3: Causal de Finetti -- multivariate
• Theorem 4: Causal de Finetti -- multivariate and categorical
• Definition 4: Acyclic directed mixed graph (ADMG) Richardson2003
• Definition 5: $\mathcal{I}$-map koller2009probabilistic
• Definition 6: Global Markov property and faithfulness zhang2012transformational