All Nodes are created Not Equal: Node-Specific Layer Aggregation and Filtration for GNN

TL;DR

The paper tackles the limitations of deep GNNs in the presence of heterophily and over-smoothing by introducing NoSAF, a Node-Specific Layer Aggregation and Filtration framework that learns per-node filters \(\gamma_i^l\) and maintains a dynamic codebank \(\mathcal{C}^l\) to control information flow across layers. A deeper variant, NoSAF-D, adds a compensatory mechanism to recover information lost during filtration, enabling ultra-deep networks. Empirical results across 17 benchmarks, including large-scale graphs OGBN-Arxiv and OGBN-Proteins, show NoSAF(-D) achieving state-of-the-art performance on both homophilic and heterophilic graphs with modest storage overhead. The work is supported by an information-bottleneck perspective that justifies improved information retention and robustness in deep GNNs, and demonstrates practical impact for scalable, versatile graph representation learning.

Abstract

The ever-designed Graph Neural Networks, though opening a promising path for the modeling of the graph-structure data, unfortunately introduce two daunting obstacles to their deployment on devices. (I) Most of existing GNNs are shallow, due mostly to the over-smoothing and gradient-vanish problem as they go deeper as convolutional architectures. (II) The vast majority of GNNs adhere to the homophily assumption, where the central node and its adjacent nodes share the same label. This assumption often poses challenges for many GNNs working with heterophilic graphs. Addressing the aforementioned issue has become a looming challenge in enhancing the robustness and scalability of GNN applications. In this paper, we take a comprehensive and systematic approach to overcoming the two aforementioned challenges for the first time. We propose a Node-Specific Layer Aggregation and Filtration architecture, termed NoSAF, a framework capable of filtering and processing information from each individual nodes. NoSAF introduces the concept of "All Nodes are Created Not Equal" into every layer of deep networks, aiming to provide a reliable information filter for each layer's nodes to sieve out information beneficial for the subsequent layer. By incorporating a dynamically updated codebank, NoSAF dynamically optimizes the optimal information outputted downwards at each layer. This effectively overcomes heterophilic issues and aids in deepening the network. To compensate for the information loss caused by the continuous filtering in NoSAF, we also propose NoSAF-D (Deep), which incorporates a compensation mechanism that replenishes information in every layer of the model, allowing NoSAF to perform meaningful computations even in very deep layers.

Figures (7)

• Figure 1: Challenges in deep GNN architectures (${\mathcal{D}}1\sim{\mathcal{D}}1$): limited node discrimination with skip-connections (the upper pipeline) and inadequate handling of heterophilic graphs (lower section).
• Figure 2: The motivation of our NoSAF framework.The pie charts depict the proportion of nodes with varying homophily in the Computers dataset ($\mathcal{H}=0.79$) and the Actor dataset ($\mathcal{H}=0.22$). In the bar charts, "+H" indicates that the model assigns $\mathcal{H}_i$ as the weight for aggregating information for each node $i$.
• Figure 3: The model overview of NoSAF and NoSAF-D version. NoSAF assigns weight ${\gamma}^l_{i}$ to each node $i$ at layer $l$, implementing hierarchical filtering. It also dynamically updates the codebank to ensure that the most optimal information is retained for output. "Filter" refers to the filtration operation as demonstrated by Equation (9).
• Figure 4: Model performance across large-scale graphs.
• Figure 5: The comparison of the smoothness of the output at each layer for all deep models. The line graph on the upper side displays the results on PubMed ($\mathcal{H}=0.79$), while the line graph on the lower side shows the results on Actor ($\mathcal{H}=0.22$).