Message Detouring: A Simple Yet Effective Cycle Representation for Expressive Graph Learning

TL;DR

This work introduces message detouring, a simple yet powerful cycle representation for expressive graph learning. By counting detour paths to form the Detour Number (DeN) and combining node-degree with Detour signals, the authors show that 1-WL initialization can attain the expressive power of higher-order WL tests at lower cost. They propose WL DeN, DeN-weighted MPNN, and a Transformer-based MDNN to integrate detour-based cycle information into graph kernels and neural networks. Empirical results on synthetic and real-world datasets demonstrate strong expressiveness and improved performance in graph and node classification, supporting the practical impact of efficiently capturing cycles in graphs.

Abstract

Graph learning is crucial in the fields of bioinformatics, social networks, and chemicals. Although high-order graphlets, such as cycles, are critical to achieving an informative graph representation for node classification, edge prediction, and graph recognition, modeling high-order topological characteristics poses significant computational challenges, restricting its widespread applications in machine learning. To address this limitation, we introduce the concept of \textit{message detouring} to hierarchically characterize cycle representation throughout the entire graph, which capitalizes on the contrast between the shortest and longest pathways within a range of local topologies associated with each graph node. The topological feature representations derived from our message detouring landscape demonstrate comparable expressive power to high-order \textit{Weisfeiler-Lehman} (WL) tests but much less computational demands. In addition to the integration with graph kernel and message passing neural networks, we present a novel message detouring neural network, which uses Transformer backbone to integrate cycle representations across nodes and edges. Aside from theoretical results, experimental results on expressiveness, graph classification, and node classification show message detouring can significantly outperform current counterpart approaches on various benchmark datasets.

Figures (7)

• Figure 1: Comparison between 1-WL initialized by our node-wise cycle representation 3-DeN (see details in Section \ref{['DeN']} Detour number) and node degree initialized 1-, 2-, or 3-WL tests. (A): An example of two non-isomorphic graphs $\mathbf{G}_1$ and $\mathbf{G}_2$. (B): Final converged coloring results of various WL tests. (C): The 3-DeN shows enough expressive power to differentiate $\mathbf{G}_1$ that has 2 detour paths (green arrows) and $\mathbf{G}_2$ that has none with respect to edge ${cd}$. In contrast, conventional WL tests fail until using 3-tuples of nodes where 3-WL test is much more computationally demanding than our detouring mechanism.
• Figure 2: The presence of cycles poses challenges to recognize isomorphism via message passing mechanism. Graphs with (a) overlapped, (b) touching, and (c) separate cycles are 1-WL equivalent when node degree is identical. (d) Topological differences between graphs can be captured by the number of detour paths, i.e. detour number (DeN). For example, the pattern of DeN on edge $cd$ is well aligned with the trending of cycle-to-cycle relationship from (a)-(c), i.e., DeN on edge $cd$ consistently decrease as the topological context between two underlying cycles (filled with different strip patterns) evolves from highly overlapped to no overlap at all. The sketch of detour paths in $\mathbf{G}_1$-$\mathbf{G}_3$ is shown in Appendix \ref{['appendix_detour_path']}.
• Figure 3: The flowchart of message detouring neural network (MDNN) with a 3-vertex graph as an example, where node and corresponding tokens framed by the same color. Norm denotes the batch normalization.
• Figure 4: Expressive power: the ratio of distinguishable graph that is non-isomorphic from pre-defined opponent of synthetic graphs (first two columns from left to right) and graphs have a different class of real-world graphs (last five columns). Each row is sorted based on computational cost. The last row is labeling nodes by 1-WL results of every 3-tuple and other rows are all labeling by a node-wise representation. The subscript of row names denotes the number of iterations, and the default is ten.
• Figure 5: Coloring results of each iteration of 1-WL tests for one example of a pair of real-world graphs from the ENZYMES dataset. Degree-initialized WL failed to distinguish this pair, while the proposed detour number (DeN) succeeded.

Theorems & Definitions (11)

• Definition 1.1: Detour path
• Definition 2.1: Detour path set
• Remark 2.2
• Remark 2.3
• Definition 3.1: Detour number
• Proposition 3.2
• Theorem 3.3
• proof
• Lemma 4.1
• proof