Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LOHA: Direct Graph Spectral Contrastive Learning Between Low-pass and High-pass Views

Ziyun Zou, Yinghui Jiang, Lian Shen, Juan Liu, Xiangrong Liu

TL;DR

LOHA addresses self-supervised graph representation learning across graphs with varying homophily by directly contrasting low-pass and high-pass spectral views. It introduces a spectral signal trend as a stable, high-dimensional composite feature to reunite node representations and derives a three-part loss that separates the views while reuniting them through spectral information. The method relies on learnable Chebyshev-based filters to form two views, then combines them with a final embedding, and optimizes with a contrastive objective plus a spectral-trend reunion term. Empirically, LOHA achieves state-of-the-art performance across 9 real-world datasets, often surpassing fully supervised models on heterophilic graphs, and ablations confirm the importance of both the view separation and reunion components. This work demonstrates that harmony in diversity, grounded in spectral properties, can enhance the efficacy of SGNNs for diverse graph structures.

Abstract

Spectral Graph Neural Networks effectively handle graphs with different homophily levels, with low-pass filter mining feature smoothness and high-pass filter capturing differences. When these distinct filters could naturally form two opposite views for self-supervised learning, the commonalities between the counterparts for the same node remain unexplored, leading to suboptimal performance. In this paper, a simple yet effective self-supervised contrastive framework, LOHA, is proposed to address this gap. LOHA optimally leverages low-pass and high-pass views by embracing "harmony in diversity". Rather than solely maximizing the difference between these distinct views, which may lead to feature separation, LOHA harmonizes the diversity by treating the propagation of graph signals from both views as a composite feature. Specifically, a novel high-dimensional feature named spectral signal trend is proposed to serve as the basis for the composite feature, which remains relatively unaffected by changing filters and focuses solely on original feature differences. LOHA achieves an average performance improvement of 2.8% over runner-up models on 9 real-world datasets with varying homophily levels. Notably, LOHA even surpasses fully-supervised models on several datasets, which underscores the potential of LOHA in advancing the efficacy of spectral GNNs for diverse graph structures.

LOHA: Direct Graph Spectral Contrastive Learning Between Low-pass and High-pass Views

TL;DR

LOHA addresses self-supervised graph representation learning across graphs with varying homophily by directly contrasting low-pass and high-pass spectral views. It introduces a spectral signal trend as a stable, high-dimensional composite feature to reunite node representations and derives a three-part loss that separates the views while reuniting them through spectral information. The method relies on learnable Chebyshev-based filters to form two views, then combines them with a final embedding, and optimizes with a contrastive objective plus a spectral-trend reunion term. Empirically, LOHA achieves state-of-the-art performance across 9 real-world datasets, often surpassing fully supervised models on heterophilic graphs, and ablations confirm the importance of both the view separation and reunion components. This work demonstrates that harmony in diversity, grounded in spectral properties, can enhance the efficacy of SGNNs for diverse graph structures.

Abstract

Spectral Graph Neural Networks effectively handle graphs with different homophily levels, with low-pass filter mining feature smoothness and high-pass filter capturing differences. When these distinct filters could naturally form two opposite views for self-supervised learning, the commonalities between the counterparts for the same node remain unexplored, leading to suboptimal performance. In this paper, a simple yet effective self-supervised contrastive framework, LOHA, is proposed to address this gap. LOHA optimally leverages low-pass and high-pass views by embracing "harmony in diversity". Rather than solely maximizing the difference between these distinct views, which may lead to feature separation, LOHA harmonizes the diversity by treating the propagation of graph signals from both views as a composite feature. Specifically, a novel high-dimensional feature named spectral signal trend is proposed to serve as the basis for the composite feature, which remains relatively unaffected by changing filters and focuses solely on original feature differences. LOHA achieves an average performance improvement of 2.8% over runner-up models on 9 real-world datasets with varying homophily levels. Notably, LOHA even surpasses fully-supervised models on several datasets, which underscores the potential of LOHA in advancing the efficacy of spectral GNNs for diverse graph structures.
Paper Structure (21 sections, 1 theorem, 16 equations, 3 figures, 3 tables)

This paper contains 21 sections, 1 theorem, 16 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Notations
  4. Graph Spectral Filtering
  5. Pre-analysis
  6. Theoretical Analysis
  7. Method
  8. Views Construction
  9. Loss Design
  10. Views Separation.
  11. Views Reunion.
  12. Experiment
  13. Baselines and Settings
  14. Evaluation Protocol.
  15. Datasets
  16. ...and 6 more sections

Key Result

Theorem 1

Given the expectation of $\mathcal{C}_i$ as $\mathbb{E}(\mathcal{C}_i$), $d_i$ as the degree of node $i$. For any $t > 0$, the probability that the distance between $\mathcal{C}_i$ and its expectation is larger than $t$ is bounded by:

Figures (3)

  • Figure 1: Initialized filters for demo experiments.
  • Figure 2: Illustration for the overall Pipeline of LOHA. Based on the natural opposite specialties between low-pass view and high-pass view, we minimize the mutual information between these two sides to construct contrastive views and separate sub-view features to learn better filters. Further, in the views reunion section, we propose to use a composite feature based on the spectral signal trends of the two views to reunite node features.
  • Figure 3: Experiments results for ablation study between LOHA and its variants. In order to better demonstrate the effect, we used the results of the full model as the denominator for each result for normalization.

Theorems & Definitions (3)

  • Definition 1: Dirichlet Energy $\mathcal{E}$
  • Definition 2: Spectral Signal Trend $\mathcal{T_\text{r}}$
  • Theorem 1