Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning

Gabriele Dominici, Pietro Barbiero, Mateo Espinosa Zarlenga, Alberto Termine, Martin Gjoreski, Giuseppe Marra, Marc Langheinrich

TL;DR

The experiments show that Causal CGMs can match the generalisation performance of causally opaque models, enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, boosting not just downstream accuracy after corrections but also the reliability of the explanations provided for specific instances.

Abstract

Causal opacity denotes the difficulty in understanding the "hidden" causal structure underlying the decisions of deep neural network (DNN) models. This leads to the inability to rely on and verify state-of-the-art DNN-based systems, especially in high-stakes scenarios. For this reason, circumventing causal opacity in DNNs represents a key open challenge at the intersection of deep learning, interpretability, and causality. This work addresses this gap by introducing Causal Concept Graph Models (Causal CGMs), a class of interpretable models whose decision-making process is causally transparent by design. Our experiments show that Causal CGMs can: (i) match the generalisation performance of causally opaque models, (ii) enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, boosting not just downstream accuracy after corrections but also the reliability of the explanations provided for specific instances, and (iii) support the analysis of interventional and counterfactual scenarios, thereby improving the model's causal interpretability and supporting the effective verification of its reliability and fairness.

Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning

TL;DR

The experiments show that Causal CGMs can match the generalisation performance of causally opaque models, enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, boosting not just downstream accuracy after corrections but also the reliability of the explanations provided for specific instances.

Abstract

Causal opacity denotes the difficulty in understanding the "hidden" causal structure underlying the decisions of deep neural network (DNN) models. This leads to the inability to rely on and verify state-of-the-art DNN-based systems, especially in high-stakes scenarios. For this reason, circumventing causal opacity in DNNs represents a key open challenge at the intersection of deep learning, interpretability, and causality. This work addresses this gap by introducing Causal Concept Graph Models (Causal CGMs), a class of interpretable models whose decision-making process is causally transparent by design. Our experiments show that Causal CGMs can: (i) match the generalisation performance of causally opaque models, (ii) enable human-in-the-loop corrections to mispredicted intermediate reasoning steps, boosting not just downstream accuracy after corrections but also the reliability of the explanations provided for specific instances, and (iii) support the analysis of interventional and counterfactual scenarios, thereby improving the model's causal interpretability and supporting the effective verification of its reliability and fairness.
Paper Structure (35 sections, 1 theorem, 15 equations, 17 figures, 11 tables)

This paper contains 35 sections, 1 theorem, 15 equations, 17 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Causal Concept Graph Models
  4. Blueprint
  5. Causal transparency in Concept Graph Models
  6. Training Causal CGMs
  7. High-dimensional endogenous representation
  8. Causal reasoning and verification with Causal CGM
  9. Experiments
  10. Key findings
  11. Discussion
  12. Conclusion
  13. Causal explainability, causal opacity, and causal discovery
  14. Derivation of Causal CGMs
  15. Proof of theorem 3.2
  16. ...and 20 more sections

Key Result

Theorem 3.2

Given a Causal Concept Graph Model $\Gamma = (\mathcal{N}, \mathcal{E}, \mathcal{F}_\theta)$, let the set of endogenous root nodes be defined as: $\text{roots}(\mathcal{G}') = \{ v_i' \in \mathcal{V}' \mid \nexists (v_j', v_i) \in \mathcal{E}_{\mathcal{G}'} \}$, and the set of children of root nodes

Figures (17)

  • Figure 1: (a) Standard DL models are black boxes in the sense that the causal structure of their mapping from raw input features (e.g., pixels of an image) to the target remains opaque. (b) In Concept Bottleneck Models (CBM), high-level human-interpretable concepts are first extracted through an encoder $g$ and then used to predict the target. Although CBMs are semantically transparent, the causal structure of the model's inference assumes a straightforward causal structure where concepts are causally independent and are all direct causes of the target. (c) In Causal Concept Graph Models (Causal CGMs), both the concepts' semantics and the inference's causal structure are transparent.
  • Figure 2: $p(v_1 \mid v_k', u_1; \theta_f)$ is equivalent to $p(v_1 \mid v_k, u_1; \theta_f)$ as they both represent the same query on the same conditional probability table.
  • Figure 3: (a) A 5-variable causal graph. (b) A ground-truth intervention fixes the error of the prediction $\hat{v_3}$ to the ground-truth label $v_3$. (c) A do-intervention sets the value of the second variable to a constant i.e., $v_2=0$. The intervention impacts $v_2$'s effects i.e., $v_{3,5}$, but does not alter $v_2$'s causes i.e., $v_{1}$. This operation can override ground-truth interventions. (d) A do-intervention on $v_3$blocks the causal effects of $v_2$ on $v_5$. As a result, intervening on $v_2$ cannot alter $v_5$ anymore.
  • Figure 4: Impact of ground-truth interventions on non-intervened nodes ($\uparrow$). Intervention on Causal CGM improves both concept and task accuracy.
  • Figure 5: Impact of ground-truth interventions on concept nodes ($\uparrow$). Intervention on CBM and CEM do not improve concept accuracy unlike Causal CGM.
  • ...and 12 more figures

Theorems & Definitions (6)

  • Definition 3.1: Causal Concept Graph Model
  • Theorem 3.2
  • Remark 3.3
  • Definition B.1: Generalised Causal Concept Graph Model
  • Definition B.2: Dissected Causal CGM
  • proof