Improving Signed Propagation for Graph Neural Networks in Multi-Class Environments

TL;DR

A new understanding of signed propagation for multi-class scenarios is provided and two novel strategies for improving signed propagation under multi-class graphs are introduced.

Abstract

Message-passing Graph Neural Networks (GNNs), which collect information from adjacent nodes achieve dismal performance on heterophilic graphs. Various schemes have been proposed to solve this problem, and propagating signed information on heterophilic edges has gained great attention. Recently, some works provided theoretical analysis that signed propagation always leads to performance improvement under a binary class scenario. However, we notice that prior analyses do not align well with multi-class benchmark datasets. This paper provides a new understanding of signed propagation for multi-class scenarios and points out two drawbacks in terms of message-passing and parameter update: (1) Message-passing: if two nodes belong to different classes but have a high similarity, signed propagation can decrease the separability. (2) Parameter update: the prediction uncertainty (e.g., conflict evidence) of signed neighbors increases during training, which can impede the stability of the algorithm. Based on the observation, we introduce two novel strategies for improving signed propagation under multi-class graphs. The proposed scheme combines calibration to secure robustness while reducing uncertainty. We show the efficacy of our theorem through extensive experiments on six benchmark graph datasets.

Figures (9)

• Figure 1: We provide an example to illustrate the distribution of node features in (a) binary and (b) multi-class scenarios. Without loss of generality, in the case of having $C$ classes, the representation can be achieved by adding $C-2$ angles to the binary class. In both cases, one can easily infer that the scale of the aggregated neighbors is always smaller than $|\mu|$, considering the Laplacian- or attention-based aggregation methods (please see Appendix B-1 for more details)
• Figure 2: We plot the Z in (a) Eq. \ref{['binary_diff']} and (b) Eq. \ref{['class_prob']} to compare the discrimination powers of signed GCN and zero-weight GCNs. The red and blue colored parts indicate the regions where signed GCN and zero-weight GCN produce superior performances, respectively
• Figure 3: Node classification accuracy on six benchmark datasets. Vanilla GCN utilizes the original graph, while the coefficient of heterophilous edges is changed to -1 in signed GCN and to 0 in zero-weight GCN, respectively. Here, signed GCN (+ calib) further employs two types of calibration, which is introduced in Section \ref{['methodology']}
• Figure 4: Visualization of parameter update using the Dirichlet distribution (central side means higher prediction uncertainty). (a) Binary class case and (b) Multi-class case. In both cases, signed messages separate the ego and neighbors. However, in (b) multi-class case, the uncertainty of neighbors ($j$ and $k$) connected with signed edges increases
• Figure 5: (Q2) Dissonance of vanilla GCN and its two variants; signed and zero-weight, which is the same as the one in Figure \ref{['status']}. The left one is Cora and the right one is the Chameleon

Theorems & Definitions (7)

• Corollary 4.1: Comparison of discrimination powers of signed GCN and zero-weight GCN
• Lemma 4.2: Signed GCN
• Lemma 4.3: Zero-weight GCN
• Corollary 4.4: Discrimination power
• Corollary 4.5: Prediction Uncertainty
• Theorem 5.1: Discrimination power after edge weight calibration
• Theorem 5.2: Reduced uncertainty