Homomorphism Counts as Structural Encodings for Graph Learning

TL;DR

The paper addresses the limited expressivity of common graph encodings in Graph Transformers and proposes motif structural encoding (MoSE), a flexible framework that encodes node structure via counts of graph homomorphisms. MoSE is parameterized by a motif family and a node weighting, and it subsumes RWSE as a special case while offering greater expressivity beyond fixed WL levels. The authors provide theoretical results linking MoSE to the Weisfeiler–Lehman hierarchy, demonstrate that MoSE can distinguish graphs that RWSE cannot, and empirically validate substantial performance gains across molecular and image-derived graph tasks, achieving state-of-the-art results on ZINC-12K and strong results on QM9 and synthetic data. The work offers a general, scalable approach to infusing graph structure into Graph Transformers with broad applicability across domains.

Abstract

Graph Transformers are popular neural networks that extend the well-known Transformer architecture to the graph domain. These architectures operate by applying self-attention on graph nodes and incorporating graph structure through the use of positional encodings (e.g., Laplacian positional encoding) or structural encodings (e.g., random-walk structural encoding). The quality of such encodings is critical, since they provide the necessary $\textit{graph inductive biases}$ to condition the model on graph structure. In this work, we propose $\textit{motif structural encoding}$ (MoSE) as a flexible and powerful structural encoding framework based on counting graph homomorphisms. Theoretically, we compare the expressive power of MoSE to random-walk structural encoding and relate both encodings to the expressive power of standard message passing neural networks. Empirically, we observe that MoSE outperforms other well-known positional and structural encodings across a range of architectures, and it achieves state-of-the-art performance on a widely studied molecular property prediction dataset.

Figures (6)

• Figure 2: The $4\times 4$ Rook's Graph (left) and the Shrikhande Graph (right) are non-isomorphic strongly regular graphs that have the same regularity parameters. RWSE produces the same vector on every vertex of both graphs whereas a simple construction of MoSE using $\mathsf{Spasm}\xspace(C_7) \cup \mathsf{Spasm}\xspace(C_8)$ homomorphism counts easily distinguishes the two graphs.
• Figure 3: Plot of the distribution of fractional domination numbers in our synthetic dataset.
• Figure 4: The 6 vertex cycle (left) and the disjoint sum of 2 triangles (right) are indistinguishable by 1-WL, but distinguishable by RWSE already after 3 steps. The RWSE vectors in the respective graphs are all the same and are given next to the graphs.
• Figure : (a) The nodes $u_1$ and $u_2$ (also, $v_1$ and $v_2$) can be distinguished by 1-WL, but not by RWSE.
• Figure : (a) The nodes $u_1$ and $u_2$ (also, $v_1$ and $v_2$) can be distinguished by 1-WL, but not by RWSE.

Theorems & Definitions (15)

• Example 1.1
• Example 1.2
• Proposition 4.1
• Proposition 4.2
• Proposition 4.3
• Proposition 4.4
• Proposition 4.5
• Theorem 4.6
• Proposition 4.7
• proof : Proof of Proposition \ref{['prop:tw']}