Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity
Jikai Jin, Vasilis Syrgkanis
TL;DR
The paper investigates identifiability in causal representation learning from multiple environments, proving that for linear causal models with a shared mixing function, the ground-truth model is identifiable up to surrounded-node ambiguity (SNA) when-environment diversity satisfies mild non-degeneracy conditions. It then provides LiNGCReL, an algorithm that provably recovers the ground-truth causal graph and latent variables up to SNA in the infinite-sample regime by combining linear ICA with a causal-structure discovery procedure leveraging effect-cancellation. Extending beyond linearity, the work shows that in nonparametric settings with single-node soft interventions, identifiability is still limited to SNA, and SNA is unavoidable under reasonable non-degeneracy and faithfulness/minimality assumptions. Collectively, the results delineate a sharp barrier to identifiability in causal representation learning under soft interventions and general environments, while offering a concrete method to achieve the best-possible identification in the linear case and guiding future exploration of non-linear and mixed-intervention settings.
Abstract
We study causal representation learning, the task of recovering high-level latent variables and their causal relationships in the form of a causal graph from low-level observed data (such as text and images), assuming access to observations generated from multiple environments. Prior results on the identifiability of causal representations typically assume access to single-node interventions which is rather unrealistic in practice, since the latent variables are unknown in the first place. In this work, we provide the first identifiability results based on data that stem from general environments. We show that for linear causal models, while the causal graph can be fully recovered, the latent variables are only identified up to the surrounded-node ambiguity (SNA) \citep{varici2023score}. We provide a counterpart of our guarantee, showing that SNA is basically unavoidable in our setting. We also propose an algorithm, \texttt{LiNGCReL} which provably recovers the ground-truth model up to SNA, and we demonstrate its effectiveness via numerical experiments. Finally, we consider general non-parametric causal models and show that the same identification barrier holds when assuming access to groups of soft single-node interventions.