Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Structure Learning For Graph Neural Networks

Yilun Zheng, Zhuofan Zhang, Ziming Wang, Xiang Li, Sitao Luan, Xiaojiang Peng, Lihui Chen

TL;DR

A new GSL framework is proposed, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs, and it is found that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance.

Abstract

To improve the performance of Graph Neural Networks (GNNs), Graph Structure Learning (GSL) has been extensively applied to reconstruct or refine original graph structures, effectively addressing issues like heterophily, over-squashing, and noisy structures. While GSL is generally thought to improve GNN performance, it often leads to longer training times and more hyperparameter tuning. Besides, the distinctions among current GSL methods remain ambiguous from the perspective of GNN training, and there is a lack of theoretical analysis to quantify their effectiveness. Recent studies further suggest that, under fair comparisons with the same hyperparameter tuning, GSL does not consistently outperform baseline GNNs. This motivates us to ask a critical question: is GSL really useful for GNNs? To address this question, this paper makes two key contributions. First, we propose a new GSL framework, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs. Second, after graph convolution, we analyze the differences in mutual information (MI) between node representations derived from the original topology and those from the newly constructed topology. Surprisingly, our empirical observations and theoretical analysis show that no matter which type of graph structure construction methods are used, after feeding the same GSL bases to the newly constructed graph, there is no MI gain compared to the original GSL bases. To fairly reassess the effectiveness of GSL, we conduct ablation experiments and find that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance. This finding encourages us to rethink the essential components in GNNs, such as self-training and structural encoding, in GNN design rather than GSL.

Rethinking Structure Learning For Graph Neural Networks

TL;DR

A new GSL framework is proposed, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs, and it is found that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance.

Abstract

To improve the performance of Graph Neural Networks (GNNs), Graph Structure Learning (GSL) has been extensively applied to reconstruct or refine original graph structures, effectively addressing issues like heterophily, over-squashing, and noisy structures. While GSL is generally thought to improve GNN performance, it often leads to longer training times and more hyperparameter tuning. Besides, the distinctions among current GSL methods remain ambiguous from the perspective of GNN training, and there is a lack of theoretical analysis to quantify their effectiveness. Recent studies further suggest that, under fair comparisons with the same hyperparameter tuning, GSL does not consistently outperform baseline GNNs. This motivates us to ask a critical question: is GSL really useful for GNNs? To address this question, this paper makes two key contributions. First, we propose a new GSL framework, which includes three steps: GSL base (the representation used for GSL) construction, new structure construction, and view fusion, to better understand the effectiveness of GSL in GNNs. Second, after graph convolution, we analyze the differences in mutual information (MI) between node representations derived from the original topology and those from the newly constructed topology. Surprisingly, our empirical observations and theoretical analysis show that no matter which type of graph structure construction methods are used, after feeding the same GSL bases to the newly constructed graph, there is no MI gain compared to the original GSL bases. To fairly reassess the effectiveness of GSL, we conduct ablation experiments and find that it is the pretrained GSL bases that enhance GNN performance, and in most cases, GSL cannot improve GNN performance. This finding encourages us to rethink the essential components in GNNs, such as self-training and structural encoding, in GNN design rather than GSL.

Paper Structure

This paper contains 33 sections, 10 equations, 9 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Graphs.
  4. Graph-aware models $\mathcal{M}^\mathcal{G}$ and graph-agnostic models $\mathcal{M}^{\neg\mathcal{G}}$.
  5. Graph Homophily.
  6. Contextual Stochastic Block Models with Homophily (CSBM-H).
  7. Mutual Information.
  8. Graph Structure Learning
  9. GSL Bases
  10. New Structure Construction
  11. View Fusion
  12. Training Mode
  13. Effectiveness of Graph Structure Learning
  14. Empirical Observations
  15. Setting
  16. ...and 18 more sections

Figures (9)

  • Figure 1: Examples of GSL that use neighbor distribution as GSL bases. The left shows a case of successful GSL, where new connections between red nodes are constructed using their GSL bases 3 green nodes and 1 blue node. The right shows a case of unsuccessful GSL, because the GSL bases of intra-class nodes are not consistent, and nodes with different classes are connected.
  • Figure 2: A framework of GSL.
  • Figure 3: Mutual information and accuracy of node classification on GSL bases $\boldsymbol{B}$, convoluted bases of old graphs $\boldsymbol{H}=\boldsymbol{\hat{A}}\boldsymbol{B}$, convoluted bases of new graphs $\boldsymbol{H'}=\boldsymbol{\hat{A}'}\boldsymbol{B}$, across varying homophily degrees. The rewriting bases $\boldsymbol{B}$ is set to node features $\boldsymbol{B}=\boldsymbol{X}$ (left) or aggregated features $\boldsymbol{B}=\boldsymbol{\hat{A}}\boldsymbol{X}$ (right).
  • Figure 4: Visualization of original graph and reconstructed graphs on Wisconsin
  • Figure 5: Influences of different GSL bases to model performance.
  • ...and 4 more figures