Probability Passing for Graph Neural Networks: Graph Structure and Representations Joint Learning

TL;DR

This work introduces a novel method called Probability Passing to refine the generated graph structure by aggregating edge probabilities of neighboring nodes based on observed graph, and names the proposed scheme as Probability Passing-based Graph Neural Network (PPGNN).

Abstract

Graph Neural Networks (GNNs) have achieved notable success in the analysis of non-Euclidean data across a wide range of domains. However, their applicability is constrained by the dependence on the observed graph structure. To solve this problem, Latent Graph Inference (LGI) is proposed to infer a task-specific latent structure by computing similarity or edge probability of node features and then apply a GNN to produce predictions. Even so, existing approaches neglect the noise from node features, which affects generated graph structure and performance. In this work, we introduce a novel method called Probability Passing to refine the generated graph structure by aggregating edge probabilities of neighboring nodes based on observed graph. Furthermore, we continue to utilize the LGI framework, inputting the refined graph structure and node features into GNNs to obtain predictions. We name the proposed scheme as Probability Passing-based Graph Neural Network (PPGNN). Moreover, the anchor-based technique is employed to reduce complexity and improve efficiency. Experimental results demonstrate the effectiveness of the proposed method.

Figures (6)

• Figure 1: A sketch of the proposed Probability Passing. The blue bar charts represent the edge probability distributions. The red node and yellow nodes represent the central node and its neighbors respectively.
• Figure 2: The interpretation of Probability Passing from the perspective of a two-step transition probability. The observed adjacency matrix can be viewed as a 0-1 probability matrix. In this example, the generated edge probability distribution is uniform, and the green nodes represent nodes that are not connected to the central node. The final probability from the central node to the target node is obtained through a special form of two-step transition probabilities generated by the adjacency matrix and the generated probability matrix.
• Figure 3: Illustration of the proposed modcel and its data flow. The input graph data entails node features $\mathbf{X}$ and graph adjacency $\mathbf{A}^{(0)}$. The entire model is divided into two parts: graph structure learning and node representations learning. In the graph structure learning process \ref{['Graph Learning']}, Probability Learning receives $\mathbf{X}$ and $\mathbf{A}^{(0)}$ to produce the probability matrix $\mathbf{P}$. Through probability passing, the probability matrix and the original adjacency matrix are fused, and finally $\mathbf{A}^{(*)}$ is obtained after sparsification. The node representations learning process in \ref{['Representation Learning']} uses GCN as message-passing method to generate predictions.
• Figure 4: Cora
• Figure 5: CiteSeer