Towards Dynamic Message Passing on Graphs

TL;DR

The paper addresses the bottlenecks of topology-reliant message passing in graph neural networks by introducing a dynamic, state-space framework that couples graph nodes with learnable pseudo nodes. A single recurrent layer drives the displacements of all embedded nodes, creating evolving spatial relations that define dynamic message pathways and enable global communication with linear space complexity $O(knn_p)$. Empirical results on eighteen benchmarks show that the proposed N^2 model outperforms strong baselines on graph and node classification while using far fewer parameters, and ablations confirm the value of dynamic proximity and the global/local MP components in mitigating over-smoothing and over-squashing. The work offers a scalable, flexible approach to graph representation learning with potential impact on large-scale graph tasks across domains.

Abstract

Message passing plays a vital role in graph neural networks (GNNs) for effective feature learning. However, the over-reliance on input topology diminishes the efficacy of message passing and restricts the ability of GNNs. Despite efforts to mitigate the reliance, existing study encounters message-passing bottlenecks or high computational expense problems, which invokes the demands for flexible message passing with low complexity. In this paper, we propose a novel dynamic message-passing mechanism for GNNs. It projects graph nodes and learnable pseudo nodes into a common space with measurable spatial relations between them. With nodes moving in the space, their evolving relations facilitate flexible pathway construction for a dynamic message-passing process. Associating pseudo nodes to input graphs with their measured relations, graph nodes can communicate with each other intermediately through pseudo nodes under linear complexity. We further develop a GNN model named $\mathtt{\mathbf{N^2}}$ based on our dynamic message-passing mechanism. $\mathtt{\mathbf{N^2}}$ employs a single recurrent layer to recursively generate the displacements of nodes and construct optimal dynamic pathways. Evaluation on eighteen benchmarks demonstrates the superior performance of $\mathtt{\mathbf{N^2}}$ over popular GNNs. $\mathtt{\mathbf{N^2}}$ successfully scales to large-scale benchmarks and requires significantly fewer parameters for graph classification with the shared recurrent layer.

Figures (13)

• Figure 1: Comparison between different connection patterns for pseudo nodes and graph nodes.
• Figure 2: Dynamic message-passing pathway construction in common state space. Graph nodes and pseudo nodes interact actively in the common state space, constructing dynamic message pathways through proximity measurement. In empirical model analysis, pseudo nodes tend to be attracted toward a distinct graph node cluster.
• Figure 3: Node classification results on small-scale heterophilic graphs (measured by ROC-AUC except accuracy for amazon-ratings: %).$\dagger$ denotes our reproduced results.
• Figure 4: Effectiveness study.
• Figure 5: Distribution of embedded nodes. The t-sne t-SNE results under different training epochs are compared. 0, 1, 2, 3, 4, 5 and 6 denote graph nodes with different labels. $\bigtriangleup$ denotes pseudo nodes. $\star$ denotes class nodes.