SF-GNN: Self Filter for Message Lossless Propagation in Deep Graph Neural Network

TL;DR

The paper addresses the degradation of deep graph neural networks beyond traditional over-smoothing/over-squashing explanations by proposing representation-interference as the root cause. It introduces SF-GNN, a plug-in framework that maintains two node representations (node and message) and a self-filter module to block propagation of low-quality signals, using a quality metric $qual_v^{(l)}$ and a gating mechanism $SFM(\mathbf{h}_v^{(l)})$ implemented via $\text{gumbel extrightarrowsoftmax}$. The approach is demonstrated across node classification on homogeneous and heterogeneous graphs and link prediction on knowledge graphs, showing improved accuracy, delayed degradation with more layers, and greater stability than state-of-the-art baselines. The findings suggest SF-GNN's broad applicability and potential to unlock deeper architectures without sacrificing performance, at the cost of additional parameters to manage gating. Overall, SF-GNN provides a concrete mechanism to mitigate propagation interference, improving deep GNN performance in both standard graph datasets and complex KG scenarios.

Abstract

Graph Neural Network (GNN), with the main idea of encoding graph structure information of graphs by propagation and aggregation, has developed rapidly. It achieved excellent performance in representation learning of multiple types of graphs such as homogeneous graphs, heterogeneous graphs, and more complex graphs like knowledge graphs. However, merely stacking GNN layers may not improve the model's performance and can even be detrimental. For the phenomenon of performance degradation in deep GNNs, we propose a new perspective. Unlike the popular explanations of over-smoothing or over-squashing, we think the issue arises from the interference of low-quality node representations during message propagation. We introduce a simple and general method, SF-GNN, to address this problem. In SF-GNN, we define two representations for each node, one is the node representation that represents the feature of the node itself, and the other is the message representation specifically for propagating messages to neighbor nodes. A self-filter module evaluates the quality of the node representation and decides whether to integrate it into the message propagation based on this quality assessment. Experiments on node classification tasks for both homogeneous and heterogeneous graphs, as well as link prediction tasks on knowledge graphs, demonstrate that our method can be applied to various GNN models and outperforms state-of-the-art baseline methods in addressing deep GNN degradation.

Figures (3)

• Figure 1: Ideal and actual MRR of R-GCN as GNN layer increases on link prediction task.
• Figure 2: The propagation of representation $\mathbf{h}_{A}^{(l)}$ ($\mathbf{m}_{A}^{(l)}$ in ours) of node A. is node with high-quality representation $\mathbf{h}_{*}^{(*)}$, is node with low-quality representation $\mathbf{h}_{*}^{(*)}$, is node receives undamaged $\mathbf{h}_{A}^{(l)}$ or $\mathbf{m}_{A}^{(l)}$, is the propagation path, represents $\mathbf{h}_{A}^{(l)}$ fails to propagate due to being damaged.
• Figure 3: Link prediction results of different GNN layers.