Differentiable Constraint-Based Causal Discovery

TL;DR

This work tackles causal discovery from observational data by introducing a differentiable notion of $d$-separation grounded in percolation theory and soft logic. The authors formulate a differentiable d-separation framework that converts discrete CI constraints into continuous, gradient-friendly objectives, and instantiate it as DAGPA (DAG Percolation Apartness) which learns weighted adjacency matrices via gradient-based optimization and discrete Langevin sampling. DAGPA directly leverages CI test p-values as soft supervision for low-order CI statements, while enforcing acyclicity through a log-determinant regularizer, thereby bridging constraint-based and score-based paradigms. Empirically, it shows strong robustness in low-sample regimes and competitive performance on the Sachs real-world dataset and synthetic benchmarks, highlighting the practical potential of differentiable percolation-based CAusal Discovery. These results suggest a promising direction for hybrid causal discovery methods that exploit CI structure with differentiable optimization, and pave the way for integrating CI-driven signals with likelihood-based score methods in future work.

Abstract

Causal discovery from observational data is a fundamental task in artificial intelligence, with far-reaching implications for decision-making, predictions, and interventions. Despite significant advances, existing methods can be broadly categorized as constraint-based or score-based approaches. Constraint-based methods offer rigorous causal discovery but are often hindered by small sample sizes, while score-based methods provide flexible optimization but typically forgo explicit conditional independence testing. This work explores a third avenue: developing differentiable $d$-separation scores, obtained through a percolation theory using soft logic. This enables the implementation of a new type of causal discovery method: gradient-based optimization of conditional independence constraints. Empirical evaluations demonstrate the robust performance of our approach in low-sample regimes, surpassing traditional constraint-based and score-based baselines on a real-world dataset. Code and data of the proposed method are publicly available at https://github$.$com/PurdueMINDS/DAGPA.

Figures (11)

• Figure 1: Illustration of the graph connectivity patterns checked by the low-order d-separation/d-connection FOL formulæ (\ref{['def:fol-d-sep']}) to determine whether the given query ($(x, y)$ or $(x, y \mid z)$) is d-separated or d-connected. The double-dashed line $= = \cdots = =$ refers to a 0th-order d-connecting path, and the arrowed line $\rightarrow \cdots \rightarrow$ refers to a directed path. In either pattern, the variable nodes $a$ and $b$ could be the query node $x$ or $y$ themselves.
• Figure 2: The Low-order CI Statement Matthews Correlation Coefficient (CI-MCC) metric measures alignment between the low-order d-separation statements predicted by the causal discovery model, and those implied from the ground-truth DAG (symmetric statements are considered).
• Figure 3: The empirical CDF of CI-MCC on Sachs sachs2005causal. The closer the curves to the bottom-right ($\searrow$), the better the performance. DAGPA achieves the best performance compared to all baseline methods.
• Figure 4: The empirical cumulative distribution functions (CDFs) of the low-order CI Statement Matthews Correlation Coefficient (CI-MCC) on two synthetic binary settings. The closer the curves to the bottom-right ($\searrow$), the better the performance. Our method achieves on-par performance to the constraint-based baselines in high-sample regime (\ref{['fig:exp-synthetic-binary-a']}) while demonstrating much better robustness in low-sample regime (\ref{['fig:exp-synthetic-binary-b']}).
• Figure 5: Example random graph to illustrate that the Max-Product Bellman-Ford reachability computes a lower bound of the true percolation-based reachability probability, whereas diffusion-based random walk computes an upper bound. Marked numbers are the edge probabilities. For the random walk computation, we assume a model with self-loop probabilities, e.g. $P(x \to x) = 0.6$ so that the transition probabilities to other nodes and the self-loop probabilities sum to one.

Theorems & Definitions (26)

• Definition 3.0
• Theorem 3.1
• Definition 3.1
• Lemma 3.1
• Definition 3.1
• Definition 3.1
• Lemma 3.1: Reachability Percolation Lower Bound
• Theorem 3.2: Lower Bound on Expected $d$-Separation Statements
• Definition 4.1: Multi-task Constraint-based CI Statement Losses of DAGPA
• Lemma 4.1: Consistency of Multi-Task CI Losses