Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Contrastive Invariant Learning from the Causal Perspective

Yanhu Mo, Xiao Wang, Shaohua Fan, Chuan Shi

TL;DR

This work analyzes graph contrastive learning through a structural causal model, identifying that standard augmentations can mix causal and non-causal graph factors and degrade invariance. It introduces GCIL, which employs spectral augmentation to intervene on non-causal content while preserving causal information and adds a dimension-wise invariance objective along with an HSIC-based independence objective to separate causal factors. The method yields state-of-the-art or competitive node classification results across five datasets, with ablations confirming the critical roles of causal intervention, invariance, and independence terms. By aligning representations to causal content and suppressing confounding, GCIL offers a principled approach to robust, invariant graph representations with practical impact for self-supervised learning on graphs.

Abstract

Graph contrastive learning (GCL), learning the node representation by contrasting two augmented graphs in a self-supervised way, has attracted considerable attention. GCL is usually believed to learn the invariant representation. However, does this understanding always hold in practice? In this paper, we first study GCL from the perspective of causality. By analyzing GCL with the structural causal model (SCM), we discover that traditional GCL may not well learn the invariant representations due to the non-causal information contained in the graph. How can we fix it and encourage the current GCL to learn better invariant representations? The SCM offers two requirements and motives us to propose a novel GCL method. Particularly, we introduce the spectral graph augmentation to simulate the intervention upon non-causal factors. Then we design the invariance objective and independence objective to better capture the causal factors. Specifically, (i) the invariance objective encourages the encoder to capture the invariant information contained in causal variables, and (ii) the independence objective aims to reduce the influence of confounders on the causal variables. Experimental results demonstrate the effectiveness of our approach on node classification tasks.

Graph Contrastive Invariant Learning from the Causal Perspective

TL;DR

This work analyzes graph contrastive learning through a structural causal model, identifying that standard augmentations can mix causal and non-causal graph factors and degrade invariance. It introduces GCIL, which employs spectral augmentation to intervene on non-causal content while preserving causal information and adds a dimension-wise invariance objective along with an HSIC-based independence objective to separate causal factors. The method yields state-of-the-art or competitive node classification results across five datasets, with ablations confirming the critical roles of causal intervention, invariance, and independence terms. By aligning representations to causal content and suppressing confounding, GCIL offers a principled approach to robust, invariant graph representations with practical impact for self-supervised learning on graphs.

Abstract

Graph contrastive learning (GCL), learning the node representation by contrasting two augmented graphs in a self-supervised way, has attracted considerable attention. GCL is usually believed to learn the invariant representation. However, does this understanding always hold in practice? In this paper, we first study GCL from the perspective of causality. By analyzing GCL with the structural causal model (SCM), we discover that traditional GCL may not well learn the invariant representations due to the non-causal information contained in the graph. How can we fix it and encourage the current GCL to learn better invariant representations? The SCM offers two requirements and motives us to propose a novel GCL method. Particularly, we introduce the spectral graph augmentation to simulate the intervention upon non-causal factors. Then we design the invariance objective and independence objective to better capture the causal factors. Specifically, (i) the invariance objective encourages the encoder to capture the invariant information contained in causal variables, and (ii) the independence objective aims to reduce the influence of confounders on the causal variables. Experimental results demonstrate the effectiveness of our approach on node classification tasks.
Paper Structure (17 sections, 14 equations, 4 figures, 3 tables)

This paper contains 17 sections, 14 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Causal Analysis on GCL
  4. Notations and Framework
  5. Causal Interpretation
  6. The Proposed Model: GCIL
  7. Causal Intervention
  8. Invariance Objective
  9. Independence Objective
  10. Optimization Objective
  11. Experiements
  12. Experimental Setup
  13. Node Classification
  14. Ablation Studies
  15. Hyper-parameter Sensitivity
  16. ...and 2 more sections

Figures (4)

  • Figure 1: SCM of the graph generation process. The dashed circle and solid circle represent unobserved and observed variables, respectively.
  • Figure 2: Overview of the GCIL framework. Given an original graph G, we first generate two views by spectral and random augmentation. The two views are subsequently fed into a shared GNN encoder to generate representations. At last, we optimize the invariance objective and the independence objective to render the model to learn the invariant representations.
  • Figure 3: The hyper-parameter sensitivity of GCIL with varying $\alpha$, $\beta$ and $\gamma$ on Cora and Wiki-CS datasets.
  • Figure 4: Correlation matrix of the representations on Wiki-CS.