Score matching through the roof: linear, nonlinear, and latent variables causal discovery
Francesco Montagna, Philipp M. Faller, Patrick Bloebaum, Elke Kirschbaum, Francesco Locatello
TL;DR
This work develops a score-based framework for causal discovery that leverages the gradient and Hessian of the log-density to identify causal structure, including in the presence of latent variables. It extends identifiability results beyond nonlinear additive-noise models, introducing AdaScore, a flexible algorithm capable of yielding a Markov equivalence class, a DAG, or a mixed graph depending on assumptions. Theoretically, the score’s Jacobian encodes m-separation information for visible variables, and under additive-noise assumptions it can identify direct edges in latent settings (nonlinear case) or ancestral relations (linear case). Empirically, AdaScore performs competitively with state-of-the-art baselines on synthetic and real datasets, scales to moderate graph sizes, and provides robust guarantees across linear, nonlinear, and latent-variable regimes, marking a step toward broadly applicable, score-based causal discovery. The work thus offers a unifying, theory-backed approach to causal discovery that can adapt to varying degrees of latent confounding and mechanism complexity, with practical implications for scalable structure learning in complex domains.
Abstract
Causal discovery from observational data holds great promise, but existing methods rely on strong assumptions about the underlying causal structure, often requiring full observability of all relevant variables. We tackle these challenges by leveraging the score function $\nabla \log p(X)$ of observed variables for causal discovery and propose the following contributions. First, we fine-tune the existing identifiability results with the score on additive noise models, showing that their assumption of nonlinearity of the causal mechanisms is not necessary. Second, we establish conditions for inferring causal relations from the score even in the presence of hidden variables; this result is two-faced: we demonstrate the score's potential to infer the equivalence class of causal graphs with hidden variables (while previous results are restricted to the fully observable setting), and we provide sufficient conditions for identifying direct causes in latent variable models. Building on these insights, we propose a flexible algorithm suited for causal discovery on linear, nonlinear, and latent variable models, which we empirically validate.