Unifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency

TL;DR

The model called Sufficiency and Necessity Inspired Graph Learning (SNIGL), which ensembles an invariant subgraph classifier on top of latent sufficient and necessary invariant subgraphs, and a domain variant subgraph classifier specific to the test domain for generalization enhancement, is devised.

Abstract

Graph Out-of-Distribution (OOD), requiring that models trained on biased data generalize to the unseen test data, has considerable real-world applications. One of the most mainstream methods is to extract the invariant subgraph by aligning the original and augmented data with the help of environment augmentation. However, these solutions might lead to the loss or redundancy of semantic subgraphs and result in suboptimal generalization. To address this challenge, we propose exploiting Probability of Necessity and Sufficiency (PNS) to extract sufficient and necessary invariant substructures. Beyond that, we further leverage the domain variant subgraphs related to the labels to boost the generalization performance in an ensemble manner. Specifically, we first consider the data generation process for graph data. Under mild conditions, we show that the sufficient and necessary invariant subgraph can be extracted by minimizing an upper bound, built on the theoretical advance of the probability of necessity and sufficiency. To further bridge the theory and algorithm, we devise the model called Sufficiency and Necessity Inspired Graph Learning (SNIGL), which ensembles an invariant subgraph classifier on top of latent sufficient and necessary invariant subgraphs, and a domain variant subgraph classifier specific to the test domain for generalization enhancement. Experimental results demonstrate that our SNIGL model outperforms the state-of-the-art techniques on six public benchmarks, highlighting its effectiveness in real-world scenarios.

Figures (4)

• Figure 1: Illustration of graph OOD methods with invariant subgraph learning, (a) Existing methods exploit either sufficient (SF) or necessary (NC) invariant features due to the unclear trade-off between invariant graph feature space constraint and prediction loss. Our method exploits necessary and sufficient (NS) invariant features that achieve the optimal trade-off, and ensembles domain variant features to further improve generalization. (b) To illustrate the concepts of these features, we provide a toy example of a dataset with distribution shifts for graph classification. (c) The first line is all SF or NC invariant features for the "house" label, where NC features may lead to two labels and SF features may not be included in each "house" graph. In the third line, the correlation between variant features and labels will change with the domain where these features are located.
• Figure 2: The causal graph for domain generalization problem on Graph DBLP:journals/corr/abs-1907-02893. Grey and white nodes denote the latent and observed variables, respectively.
• Figure 3: Ablation studies on four datasets. We explore the impact of the key components of our SNIGL, i.e, PNS risk and the ensemble strategy
• Figure 4: Visualization of molecule examples in the OGBG-HIV dataset. Nodes with different colors denote different atoms, and edges denote different chemical bonds.

Theorems & Definitions (9)

• Definition 1
• Definition 2
• Definition 3
• Theorem 1
• Theorem 2
• proof
• Lemma 1
• proof
• proof