What Are Good Positional Encodings for Directed Graphs?

TL;DR

This work addresses the challenge of designing positional encodings for directed graphs by introducing Walk Profile, a bidirectional walk descriptor $oldsymbol{ extPhi}_{u,v}(oldsymbol{ extell}, oldsymbol{k})$ that captures forward/backward edge counts. It shows that existing PEs, including Mag-PE and SVD-PE, cannot fully express these profiles, and proposes Multi-q Magnetic Laplacian PE (Multi-q Mag-PE) which uses multiple frequencies $oldsymbol{q}=(q_1,\, o, q_Q)$ to enable exact reconstruction of walks up to length $L$ via a linear system $m{F}oldsymbol{ extPhi}=oldsymbol{Y}$. A basis-invariant stable neural architecture for processing complex eigenvectors is developed to address basis ambiguity and instability in Mag-PEs, unifying Laplacian and magnetic approaches. Experimental results across distance prediction, sorting network satisfiability, and circuit property prediction demonstrate consistent performance gains when using Multi-q Mag-PE with stable processing, with manageable runtime trade-offs and publicly available code. The proposed framework provides a principled, expressive, and stable way to inject directed graph structure into graph transformers and related models.

Abstract

Positional encodings (PEs) are essential for building powerful and expressive graph neural networks and graph transformers, as they effectively capture the relative spatial relationships between nodes. Although extensive research has been devoted to PEs in undirected graphs, PEs for directed graphs remain relatively unexplored. This work seeks to address this gap. We first introduce the notion of Walk Profile, a generalization of walk-counting sequences for directed graphs. A walk profile encompasses numerous structural features crucial for directed graph-relevant applications, such as program analysis and circuit performance prediction. We identify the limitations of existing PE methods in representing walk profiles and propose a novel Multi-q Magnetic Laplacian PE, which extends the Magnetic Laplacian eigenvector-based PE by incorporating multiple potential factors. The new PE can provably express walk profiles. Furthermore, we generalize prior basis-invariant neural networks to enable the stable use of the new PE in the complex domain. Our numerical experiments validate the expressiveness of the proposed PEs and demonstrate their effectiveness in solving sorting network satisfiability and performing well on general circuit benchmarks. Our code is available at https://github.com/Graph-COM/Multi-q-Maglap.

Figures (11)

• Figure 1: Examples of real-world directed motifs/patterns (left) and walk profile (right).
• Figure 2: Multi-q Magnetic Laplacian under stable PE framework. Eigenvectors and eigenvalues of each Magnetic Laplacian with different $q$ will be processed independently and identically and concatenated in the end.
• Figure 2: Test F1 scores over 5 random seeds for sorting network satisfiability.
• Figure 3: Test F1 scores w.r.t. different sorting network lengths.
• Figure 4: Runtime of preprocessing (left), training (middle) and inference (right) on three datasets. The shown runtime of distance dataset is 10 times the actual ones for better illustration.

Theorems & Definitions (11)

• Definition 4.1: Bidirectional Walk
• Definition 4.2: Walk Profile
• Remark 4.1
• Theorem 4.1
• Remark 4.2
• Theorem 4.2
• proof : Proof Sketch
• Theorem A.1
• proof
• Theorem A.1