Learning Efficient Positional Encodings with Graph Neural Networks

TL;DR

PEARL introduces learnable, graph-based positional encodings (PEs) that are generated by GNNs and aggregated to yield permutation-equivariant node representations. By treating GNNs as nonlinear mappings of graph spectral components and introducing random (R-PEARL) or basis (B-PEARL) node attributes, the framework achieves high expressivity and stability with linear or near-linear scaling, addressing the four key PE criteria: expressive power, stability, scalability, and genericness. The authors provide rigorous sample complexity and stability analyses, linking PEARL to existing eigenvector-based encodings while offering substantial reductions in computational cost. Empirical results across graph classification, molecular regression, OOD drug datasets, and large RelBench tasks demonstrate that PEARL variants outperform lightweight eigenvector-based methods and rival full eigenvector encodings in performance, with markedly improved efficiency. The work advances practical graph representation learning by enabling powerful, scalable PEs that generalize across graph sizes and domains.

Abstract

Positional encodings (PEs) are essential for effective graph representation learning because they provide position awareness in inherently position-agnostic transformer architectures and increase the expressive capacity of Graph Neural Networks (GNNs). However, designing powerful and efficient PEs for graphs poses significant challenges due to the absence of canonical node ordering and the scale of the graph. {In this work, we identify four key properties that graph PEs should satisfy}: stability, expressive power, scalability, and genericness. We find that existing eigenvector-based PE methods often fall short of jointly satisfying these criteria. To address this gap, we introduce PEARL, a novel framework of learnable PEs for graphs. Our primary insight is that message-passing GNNs function as nonlinear mappings of eigenvectors, enabling the design of GNN architectures for generating powerful and efficient PEs. A crucial challenge lies in initializing node attributes in a manner that is both expressive and permutation equivariant. We tackle this by initializing GNNs with random node inputs or standard basis vectors, thereby unlocking the expressive power of message-passing operations, while employing statistical pooling functions to maintain permutation equivariance. Our analysis demonstrates that PEARL approximates equivariant functions of eigenvectors with linear complexity, while rigorously establishing its stability and high expressive power. Experimental evaluations show that PEARL outperforms lightweight versions of eigenvector-based PEs and achieves comparable performance to full eigenvector-based PEs, but with one or two orders of magnitude lower complexity. Our code is available at https://github.com/ehejin/Pearl-PE.

Figures (2)

• Figure 1: PEARL framework: The input graph undergoes anonymization by removing its node and edge attributes. For each node, a set of $M$ random or basis attributes is generated. Each sample is then independently processed by a message-passing GNN, and a pooling function $\rho$ is applied to produce equivariant PEs. The graph structure, together with the generated PEs and any node or graph attributes, is subsequently processed using either a GNN or a Graph Transformer.
• Figure 2: Ablation studies on the sample size for R-PEARL; It converges with only a few samples.

Theorems & Definitions (16)

• Proposition 3.1: GNNs are nonlinear functions of eigenvectors
• Theorem 3.1: Basis Universality
• Theorem 4.3: Sample Complexity
• Corollary 4.4: Structure Counting
• Corollary 4.5: Expressive Power
• Corollary 4.6: Stability
• Remark 5.1: Expressivity, Stability
• Remark 5.2
• Lemma C.1
• Theorem D.1: Eigenvalue Universality