Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discovering Invariant Rationales for Graph Neural Networks

Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, Tat-Seng Chua

TL;DR

This paper tackles the problem of brittle, bias-driven rationales in graph neural networks by proposing DIR, a framework that discovers invariant causal rationales through interventions on training distributions. DIR combines a rationale generator, distribution intervener, a graph encoder, and two classifiers to separate causal from non-causal subgraphs and enforce stability of the C → Y relation across interventional environments. The method yields superior generalization and stronger intrinsic interpretability compared with ERM and invariant baselines on synthetic and real graph-classification tasks, with ablations showing the value of variance regularization and interventional data augmentation. These findings suggest that environment-invariant rationales can more reliably reflect true task-relevant structure and improve robustness to distribution shifts in graphs.

Abstract

Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs. It conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that are unstable. Experiments on both synthetic and real-world datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.

Discovering Invariant Rationales for Graph Neural Networks

TL;DR

This paper tackles the problem of brittle, bias-driven rationales in graph neural networks by proposing DIR, a framework that discovers invariant causal rationales through interventions on training distributions. DIR combines a rationale generator, distribution intervener, a graph encoder, and two classifiers to separate causal from non-causal subgraphs and enforce stability of the C → Y relation across interventional environments. The method yields superior generalization and stronger intrinsic interpretability compared with ERM and invariant baselines on synthetic and real graph-classification tasks, with ablations showing the value of variance regularization and interventional data augmentation. These findings suggest that environment-invariant rationales can more reliably reflect true task-relevant structure and improve robustness to distribution shifts in graphs.

Abstract

Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs. It conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that are unstable. Experiments on both synthetic and real-world datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.
Paper Structure (35 sections, 3 theorems, 18 equations, 10 figures, 7 tables, 1 algorithm)

This paper contains 35 sections, 3 theorems, 18 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Invariant Rationale Discovery
  3. Causal View of Data-Generating Process
  4. Task Formalization of Invariant Rationalization
  5. Principle & Learning Strategy of DIR
  6. DIR-Guided Implementation of Intrinsically-Interpretable GNNs
  7. Experiments
  8. Settings
  9. Main Results (RQ1)
  10. In-Depth Study (RQ2)
  11. Related Works
  12. Conclusion & Future Work
  13. Notations & Algorithm
  14. Instantiated Causal Graphs
  15. Theory
  16. ...and 20 more sections

Key Result

Theorem 1

Suppose $S\rightarrow C$ does not exist, then the oracle function $f_{Y}$ satisfies the DIR Principle (where $C$ is given) over every distribution $\tilde{\mathcal{P}} \in \mathcal{P}(G, Y)$.

Figures (10)

  • Figure 1: Base Distribution of House Motif.
  • Figure 2: (a) Causal view of data-generating process; (b) Illustration of interventional distributions.
  • Figure 3: DIR Implementation on GNNs, which includes a rationale generator, a distribution intervener, an encoder and two classifiers. For the inference, we only use $\hat{y}_{\tilde{c}}$ as the prediction.
  • Figure 4: Visualization of DIR Rationales. Each graph shows a comment, e.g., "a majestic achievement, an epic of astonishing grandeur" in (a), where rationales are highlighted by deep colors.
  • Figure 5: Two-stage Training Dynamics of DIR.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1: DIR Principle
  • Theorem 1: Necessity
  • Theorem 2: Uniqueness
  • Corollary 1: Necessity and Sufficiency