Contextualized Messages Boost Graph Representations

TL;DR

This work tackles the expressivity gap of GNNs when node feature spaces are uncountable by introducing a soft-injective framework based on pseudometrics. It derives a practical instantiation, the soft-isomorphic relational GCN (SIR-GCN), which uses anisotropic and dynamic message functions to contextualize neighborhood features with a single aggregator while maintaining computational efficiency. The authors establish theoretical connections between SIR-GCN and classical GNNs (GCN, GraphSAGE, GAT, GIN, PNA) and relate it to the 1-WL test, arguing that SIR-GCN achieves comparable expressivity under uncountable feature regimes. Empirically, SIR-GCN demonstrates strong performance on synthetic tasks and large-scale benchmarks, particularly excelling on datasets with uncountable node features, and generally outperforms several baselines with a simpler architecture. The results underscore the value of contextualized, soft-injectivity-based messaging for enhancing GNN representational power in practical graph learning tasks.

Abstract

Graph neural networks (GNNs) have gained significant attention in recent years for their ability to process data that may be represented as graphs. This has prompted several studies to explore their representational capability based on the graph isomorphism task. Notably, these works inherently assume a countable node feature representation, potentially limiting their applicability. Interestingly, only a few study GNNs with uncountable node feature representation. In the paper, a new perspective on the representational capability of GNNs is investigated across all levels$\unicode{x2014}$node-level, neighborhood-level, and graph-level$\unicode{x2014}$when the space of node feature representation is uncountable. Specifically, the injective and metric requirements of previous works are softly relaxed by employing a pseudometric distance on the space of input to create a soft-injective function such that distinct inputs may produce similar outputs if and only if the pseudometric deems the inputs to be sufficiently similar on some representation. As a consequence, a simple and computationally efficient soft-isomorphic relational graph convolution network (SIR-GCN) that emphasizes the contextualized transformation of neighborhood feature representations via anisotropic and dynamic message functions is proposed. Furthermore, a mathematical discussion on the relationship between SIR-GCN and key GNNs in literature is laid out to put the contribution into context, establishing SIR-GCN as a generalization of classical GNN methodologies. To close, experiments on synthetic and benchmark datasets demonstrate the relative superiority of SIR-GCN, outperforming comparable models in node and graph property prediction tasks.

Figures (5)

• Figure 1: A soft-injective feature map $g: \mathcal{H} \rightarrow \mathcal{S}$ corresponding to a pseudometric$d$ on $\mathcal{H}$.
• Figure 2: Hash functions $G_u$ under different message functions $g_u$.
• Figure 3: SIR-GCN generalizes classical GNNs.
• Figure 4: DictionaryLookup.
• Figure 5: HeteroEdgeCount.

Theorems & Definitions (13)

• definition 1: Pseudometric
• definition 2: Conditionally positive definite kernel scholkopf2000kernel
• remark 1
• theorem 1
• corollary 1
• theorem 2: Hilbert space representation of conditionally positive definite kernels schoenberg1938metricscholkopf2000kernelberg1984harmonic
• proof
• theorem 2
• proof
• theorem 3