Identifying Weight-Variant Latent Causal Models

TL;DR

The paper addresses identifiability in causal representation learning, highlighting transitivity as the central obstruction to recovering latent causal variables from observations. It introduces weight-variant linear Gaussian models where the causal weights and latent noise distributions are modulated by an auxiliary variable $\mathbf{u}$, and proves identifiability of latent variables up to trivial permutation and scaling under a set of assumptions and a reference environment where causal influences vanish. The authors further establish partial identifiability when not all weights vary and present SuaVE, a Structural caUsAl Variational autoEncoder that jointly learns latent causal representations, their graph, and the mapping to observed data, with a prior that enforces the weight-variant structure and a variational objective augmented by penalties to satisfy key assumptions. Empirical results on synthetic data, image-like chemistry data, and fMRI data validate the theory and show SuaVE outperforms baseline identifiable and non-identifiable methods in recovering latent causal representations and causal skeletons, demonstrating practical utility for learning robust, causally meaningful latent representations under distribution shifts.

Abstract

The task of causal representation learning aims to uncover latent higher-level causal variables that affect lower-level observations. Identifying the true latent causal variables from observed data, while allowing instantaneous causal relations among latent variables, remains a challenge, however. To this end, we start with the analysis of three intrinsic indeterminacies in identifying latent variables from observations: transitivity, permutation indeterminacy, and scaling indeterminacy. We find that transitivity acts as a key role in impeding the identifiability of latent causal variables. To address the unidentifiable issue due to transitivity, we introduce a novel identifiability condition where the underlying latent causal model satisfies a linear-Gaussian model, in which the causal coefficients and the distribution of Gaussian noise are modulated by an additional observed variable. Under certain assumptions, including the existence of a reference condition under which latent causal influences vanish, we can show that the latent causal variables can be identified up to trivial permutation and scaling, and that partial identifiability results can still be obtained when this reference condition is violated for a subset of latent variables. Furthermore, based on these theoretical results, we propose a novel method, termed Structural caUsAl Variational autoEncoder (SuaVE), which directly learns causal representations and causal relationships among them, together with the mapping from the latent causal variables to the observed ones. Experimental results on synthetic and real data demonstrate the identifiability and consistency results and the efficacy of SuaVE in learning causal representations.

Figures (25)

• Figure 1: Latent Causal Models with $\mathbf{u}$.
• Figure 2: Two equivalent structures.
• Figure 3: A Causal Fully-connected Graph, based on pre-defined causal order $z_1,z_2,z_3,z_4$, thanks to permutation indeterminacy in latent space.
• Figure 4: The proposed Latent Causal Models. Here we use the 'red' lines to indicate the changes of weights across $\mathbf{u}$, which is different from the definition of edges (i.e., causal direction) in standard causal model graphs.
• Figure 5: Implementation framework to learn linear Gaussian causal representations.

Theorems & Definitions (5)

• Theorem 1
• Corollary 2
• Corollary 3
• Theorem 4
• Definition 5