Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Contrastive Learning under Heterophily via Graph Filters

Wenhan Yang, Baharan Mirzasoleiman

TL;DR

The paper tackles graph contrastive learning on graphs with heterophily, where neighboring nodes may belong to different classes. It introduces HLCL, a dual-filter self-supervised framework that first splits the graph into homophilic and heterophilic subgraphs based on feature similarity, then applies low-pass filtering to the homophilic part and high-pass filtering to the heterophilic part before contrasting their augmented views with a shared encoder. The authors provide theoretical analysis showing HLCL encodes both low- and high-frequency information and demonstrate empirical gains over state-of-the-art CL methods and competitive performance with supervised methods on heterophilic datasets, with scalability to large graphs. They also conduct extensive ablations to justify the dual-filter design and discuss limitations, including scenarios where feature signals fail to discriminate labels.

Abstract

Graph contrastive learning (CL) methods learn node representations in a self-supervised manner by maximizing the similarity between the augmented node representations obtained via a GNN-based encoder. However, CL methods perform poorly on graphs with heterophily, where connected nodes tend to belong to different classes. In this work, we address this problem by proposing an effective graph CL method, namely HLCL, for learning graph representations under heterophily. HLCL first identifies a homophilic and a heterophilic subgraph based on the cosine similarity of node features. It then uses a low-pass and a high-pass graph filter to aggregate representations of nodes connected in the homophilic subgraph and differentiate representations of nodes in the heterophilic subgraph. The final node representations are learned by contrasting both the augmented high-pass filtered views and the augmented low-pass filtered node views. Our extensive experiments show that HLCL outperforms state-of-the-art graph CL methods on benchmark datasets with heterophily, as well as large-scale real-world graphs, by up to 7%, and outperforms graph supervised learning methods on datasets with heterophily by up to 10%.

Graph Contrastive Learning under Heterophily via Graph Filters

TL;DR

The paper tackles graph contrastive learning on graphs with heterophily, where neighboring nodes may belong to different classes. It introduces HLCL, a dual-filter self-supervised framework that first splits the graph into homophilic and heterophilic subgraphs based on feature similarity, then applies low-pass filtering to the homophilic part and high-pass filtering to the heterophilic part before contrasting their augmented views with a shared encoder. The authors provide theoretical analysis showing HLCL encodes both low- and high-frequency information and demonstrate empirical gains over state-of-the-art CL methods and competitive performance with supervised methods on heterophilic datasets, with scalability to large graphs. They also conduct extensive ablations to justify the dual-filter design and discuss limitations, including scenarios where feature signals fail to discriminate labels.

Abstract

Graph contrastive learning (CL) methods learn node representations in a self-supervised manner by maximizing the similarity between the augmented node representations obtained via a GNN-based encoder. However, CL methods perform poorly on graphs with heterophily, where connected nodes tend to belong to different classes. In this work, we address this problem by proposing an effective graph CL method, namely HLCL, for learning graph representations under heterophily. HLCL first identifies a homophilic and a heterophilic subgraph based on the cosine similarity of node features. It then uses a low-pass and a high-pass graph filter to aggregate representations of nodes connected in the homophilic subgraph and differentiate representations of nodes in the heterophilic subgraph. The final node representations are learned by contrasting both the augmented high-pass filtered views and the augmented low-pass filtered node views. Our extensive experiments show that HLCL outperforms state-of-the-art graph CL methods on benchmark datasets with heterophily, as well as large-scale real-world graphs, by up to 7%, and outperforms graph supervised learning methods on datasets with heterophily by up to 10%.
Paper Structure (28 sections, 4 theorems, 28 equations, 3 figures, 12 tables, 1 algorithm)

This paper contains 28 sections, 4 theorems, 28 equations, 3 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Graph CL under Homophily
  5. High-pass and Low-pass graph filters
  6. Graph CL under Heterophily
  7. High-pass & Low-pass graph CL (HLCL)
  8. Theoretical Analysis
  9. Experiments
  10. Results
  11. Ablation Studies
  12. HLCL with Explicit Augmentation
  13. HLCL with Single Graph Filter
  14. Subgraph Update Interval
  15. Homophily ratio can guide Tuning
  16. ...and 13 more sections

Key Result

Theorem 1

Under the above assumptions and given ideal subgraphs $G_{\text{hom}}$ and $G_{\text{het}}$, the HLCL loss can be lower-bounded as follows: where $\lambda_{\pmb{A}^{hom}}, \lambda_{\tilde{\pmb{A}}_{i}^{hom}}$ denote the eigenvalues of the low-pass filters corresponding to augmented homophilic subgraph, $\lambda_{\pmb{L}^{het}}, \lambda_{\tilde{\pmb{L}}_{i}^{het}}$ denote the eigenvalues of the hi

Figures (3)

  • Figure 1: HLCL identifies a homophilic and a heterophilic subgraph $\mathcal{G}^{hom}, \mathcal{G}^{het}$, and generates two augmentations for each subgraph. Then, it applies low-pass filters $\pmb{F}_{LP}, \tilde{\pmb{F}}_{LP}$ to the augmented homophilic subgraphs and high-pass filters $\pmb{F}_{HP}, \tilde{\pmb{F}}_{HP}$ to the augmented heterophilic subgraphs, to generate low-pass $\pmb{H}_L, \tilde{\pmb{H}}_L$ and high-pass $\pmb{H}_H, \tilde{\pmb{H}}_H$ filtered views, using the same encoder $\pmb{W}$. HLCL learns the final representations by contrasting the projected low-pass filtered augmented views $\pmb{z}^L, \tilde{\pmb{z}}^L$ and the high-pass filtered augmented views $\pmb{z}^h, \tilde{\pmb{z}}^h$ of every node.
  • Figure 2: Chameleon ($\beta\!\!=\!\!0.23$). Heterophilic graphs contain neighborhoods with homogeneous & heterogeneous labels.
  • Figure 3: GRACE vs HLCL representations. (a), (c) distribution of eigenvalues in the representation matrix. (b), (d) alignment of the labels with the eigenvectors of the representation matrix. HLCL produces higher quality representations with lower rank and higher alignment with the label vector.

Theorems & Definitions (8)

  • Theorem 1: HLCL: Spectral Invariance
  • Lemma 1
  • proof
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof