Customizing Graph Neural Networks using Path Reweighting

TL;DR

The proposed CustomGNN can automatically learn the high-level semantics for specific downstream tasks to highlight semantically relevant paths as well to filter out task-irrelevant noises in a graph.

Abstract

Graph Neural Networks (GNNs) have been extensively used for mining graph-structured data with impressive performance. However, because these traditional GNNs do not distinguish among various downstream tasks, embeddings embedded by them are not always effective. Intuitively, paths in a graph imply different semantics for different downstream tasks. Inspired by this, we design a novel GNN solution, namely Customized Graph Neural Network with Path Reweighting (CustomGNN for short). Specifically, the proposed CustomGNN can automatically learn the high-level semantics for specific downstream tasks to highlight semantically relevant paths as well to filter out task-irrelevant noises in a graph. Furthermore, we empirically analyze the semantics learned by CustomGNN and demonstrate its ability to avoid the three inherent problems in traditional GNNs, i.e., over-smoothing, poor robustness, and overfitting. In experiments with the node classification task, CustomGNN achieves state-of-the-art accuracies on three standard graph datasets and four large graph datasets. The source code of the proposed CustomGNN is available at \url{https://github.com/cjpcool/CustomGNN}.

Figures (6)

• Figure 1: Examples to illustrate the intuition of path re-weighting. (a) For node classification, the paths with intra-class nodes should be emphasized (blue paths) while paths with inter-class nodes (orange paths) should be de-emphasized. (b) For medical Q&A task, each path is a piece of knowledge. Only the Q&A task-relevant paths should be emphasized. (c) Traditional GNNs aggregate neighbors order by order, neglecting the difference between paths.
• Figure 2: The overall architectural diagram of CustomGNN. Here, $\otimes$ denotes matrix multiplication operation. There are two paths in CustomGNN. The first path highlighted in blue uses random walk to generate sub-paths (a subgraph), then, the path re-weighting module is utilized in these sub-paths for generating customized feature $\hat{\mathbf{F}}$. The orange line indicates that we utilize a simple multi-hop GNN to generate general feature $\dot{\mathbf{F}}$. Finally, we feed the aggregated feature $\{\overline{\mathbf{F}} \mid \overline{\mathbf{F}} = \hat{\mathbf{F}} \oplus \dot{\mathbf{F}}\}$ into classifier to obtain the predictions.
• Figure 3: (a) The visualization of re-weighted path attentions on Citeseer. We filter out some noisy points whose weight is less than 0.15. (b) The visualization of relevance between different categories on Citeseer. The weight on $(i,j)$ means how much the $i^{th}$ category is influenced by $j^{th}$ category.
• Figure 4: (a), (b) and (c), (d) are the mean correlations between weight and two properties (length and diversity) of paths on Cora and Citeseer respectively. In (b) and (d), to remove the influence of length, the path length is fixed on 10. Y-axis denotes the average weight of the corresponding paths.
• Figure 5: (a) is the loss of CustomGNN without path re-weighting, and (b) is the loss of CustomGNN. X-axis denotes training epochs, Y-axis denotes loss values. (c) illustrates the gap between train loss and validation loss of (a) and (b), Y-axis denotes the gap value.