Inducing Causal Structure for Interpretable Neural Networks

TL;DR

This work introduces Interchange Intervention Training (IIT), a differentiable framework that enforces a symbolic causal model to govern neural computations by aligning internal representations and enforcing counterfactual consistency via interchange interventions. IIT yields models that not only generalize better across challenging tasks but also more faithfully realize the target causal structure, as demonstrated in MNIST-PVR, ReaSCAN, and MQNLI. The method provides a formal notion of causal abstraction and a practical training objective that can be combined with multi-task learning and data augmentation. Collectively, IIT offers a principled approach to embedding interpretable, high-level causal structure into neural networks with measurable interpretability via IntInvAcc.

Abstract

In many areas, we have well-founded insights about causal structure that would be useful to bring into our trained models while still allowing them to learn in a data-driven fashion. To achieve this, we present the new method of interchange intervention training (IIT). In IIT, we (1) align variables in a causal model (e.g., a deterministic program or Bayesian network) with representations in a neural model and (2) train the neural model to match the counterfactual behavior of the causal model on a base input when aligned representations in both models are set to be the value they would be for a source input. IIT is fully differentiable, flexibly combines with other objectives, and guarantees that the target causal model is a causal abstraction of the neural model when its loss is zero. We evaluate IIT on a structural vision task (MNIST-PVR), a navigational language task (ReaSCAN), and a natural language inference task (MQNLI). We compare IIT against multi-task training objectives and data augmentation. In all our experiments, IIT achieves the best results and produces neural models that are more interpretable in the sense that they more successfully realize the target causal model.

Figures (5)

• Figure 1: Interchange intervention training example. Network $\mathcal{N}_{\land}$ performs boolean conjunction with perfect accuracy, or, equivalently, it agrees with $\mathcal{C}_{\land}$ on the four possible inputs (figure \ref{['fig:simplestart']}). However, $\mathcal{C}_{\land}$ is not a causal abstraction of $\mathcal{N}_{\land}$ under this alignment, because there are aligned interchange interventions that result in $\mathcal{N}_{\land}$ and $\mathcal{C}_{\land}$ producing different outputs, meaning that the internal dynamics of the network do not realize the structure of the causal model. To quantify this, we note that the interchange intervention accuracy (Eqn. \ref{['eq:acc']}) is $81.25\%$. After a single interchange intervention training update (figure \ref{['fig:simpletrain']}, figure \ref{['fig:simplecode']}), this is fixed: all aligned interchange interventions result in the same output (the interchange intervention accuracy is now $1$), so $\mathcal{C}_{\land}$ has become a causal abstraction of $\mathcal{N}_{\land}$ (figure \ref{['fig:simpleend']}).
• Figure 2: An illustration of an IIT update where a neural network (right) is trained to realize a causal model (left) that solves the PVR-MNIST task. Solid lines are feed-forward connections, dashed lines are interchange interventions, red lines are the flow of backpropagation. Observe that when backpropagation reaches the interchange intervention, it flows into both the source input's computation graph and the base input's graph, updating the weights below the interchange intervention twice.
• Figure 3:
• Figure 4: Performance of a pretrained BERT natural language inference model fine-tuned on the MQNLI dataset with the causal model $C^{\text{QP}_{\text{Obj}}}_{\text{NatLog}}$ from geiger-etal-2020-neural. We report the results on the evaluation set. While data augmentation leads to consistently excellent behavior accuracy (left) panel, it has very low interchange intervention accuracy. In other words, IIT is necessary for an interpretable model with high-performance.
• Figure 5: ReaSCAN examples with varying command patterns. The navigation commands and the target action sequences are in the grey boxes and green boxes respectively.