Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Diffusion Networks for Semi-supervised Learning

Wei Ye, Zexi Huang, Yunqi Hong, Ambuj Singh

TL;DR

The paper tackles the limitations of GCNs in semi-supervised learning on sparsely labeled graphs, specifically under- and over-smoothing. It introduces Graph Neural Diffusion Networks (GND-Nets), which use neural diffusions to aggregate local and global neighborhood information within a single shallow layer, guided by a power-iteration interpretation. The key contributions are the neural diffusions themselves, their adaptive, dataset-specific weighting, and extensive empirical validation showing superior performance over a wide range of baselines on multiple sparse-graph benchmarks. The work offers a practical, efficient approach for improved semi-supervised learning on graphs and points to future extensions to RNN and Transformer architectures.

Abstract

Graph Convolutional Networks (GCN) is a pioneering model for graph-based semi-supervised learning. However, GCN does not perform well on sparsely-labeled graphs. Its two-layer version cannot effectively propagate the label information to the whole graph structure (i.e., the under-smoothing problem) while its deep version over-smoothens and is hard to train (i.e., the over-smoothing problem). To solve these two issues, we propose a new graph neural network called GND-Nets (for Graph Neural Diffusion Networks) that exploits the local and global neighborhood information of a vertex in a single layer. Exploiting the shallow network mitigates the over-smoothing problem while exploiting the local and global neighborhood information mitigates the under-smoothing problem. The utilization of the local and global neighborhood information of a vertex is achieved by a new graph diffusion method called neural diffusions, which integrate neural networks into the conventional linear and nonlinear graph diffusions. The adoption of neural networks makes neural diffusions adaptable to different datasets. Extensive experiments on various sparsely-labeled graphs verify the effectiveness and efficiency of GND-Nets compared to state-of-the-art approaches.

Graph Neural Diffusion Networks for Semi-supervised Learning

TL;DR

The paper tackles the limitations of GCNs in semi-supervised learning on sparsely labeled graphs, specifically under- and over-smoothing. It introduces Graph Neural Diffusion Networks (GND-Nets), which use neural diffusions to aggregate local and global neighborhood information within a single shallow layer, guided by a power-iteration interpretation. The key contributions are the neural diffusions themselves, their adaptive, dataset-specific weighting, and extensive empirical validation showing superior performance over a wide range of baselines on multiple sparse-graph benchmarks. The work offers a practical, efficient approach for improved semi-supervised learning on graphs and points to future extensions to RNN and Transformer architectures.

Abstract

Graph Convolutional Networks (GCN) is a pioneering model for graph-based semi-supervised learning. However, GCN does not perform well on sparsely-labeled graphs. Its two-layer version cannot effectively propagate the label information to the whole graph structure (i.e., the under-smoothing problem) while its deep version over-smoothens and is hard to train (i.e., the over-smoothing problem). To solve these two issues, we propose a new graph neural network called GND-Nets (for Graph Neural Diffusion Networks) that exploits the local and global neighborhood information of a vertex in a single layer. Exploiting the shallow network mitigates the over-smoothing problem while exploiting the local and global neighborhood information mitigates the under-smoothing problem. The utilization of the local and global neighborhood information of a vertex is achieved by a new graph diffusion method called neural diffusions, which integrate neural networks into the conventional linear and nonlinear graph diffusions. The adoption of neural networks makes neural diffusions adaptable to different datasets. Extensive experiments on various sparsely-labeled graphs verify the effectiveness and efficiency of GND-Nets compared to state-of-the-art approaches.
Paper Structure (12 sections, 1 theorem, 13 equations, 1 figure, 7 tables)

This paper contains 12 sections, 1 theorem, 13 equations, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Graph Convolutional Networks
  4. Graph Diffusions
  5. Graph Neural Diffusion Networks
  6. Local and Global Neighborhood Information
  7. Neural Diffusions
  8. Experimental Evaluation
  9. Experimental Setup
  10. Results
  11. Classification
  12. Conclusion

Key Result

Theorem 1

If the graph underlying $\mathbf{\widetilde{W}}$ is non-bipartite, the vector $\mathbf{\widetilde{W}}^k\mathbf{z}$ converges and the limit is the dominant eigenvector of $\mathbf{\widetilde{W}}$.

Figures (1)

  • Figure 1: The t-SNE visualization of the random projection of the feature matrix of dataset Cora. Power iteration reveals the cluster structures after $k=19$ iterations. Colors denote classes.

Theorems & Definitions (2)

  • Theorem 1
  • Proof 1