Descriptive Kernel Convolution Network with Improved Random Walk Kernel

TL;DR

Graph kernels historically offered interpretable features but lacked learnability, while modern GNNs provide learnable representations. This work introduces $RWK^+$, an improved color-matching random walk kernel with efficient computation, and builds a descriptive unsupervised model $RWK^+$CN that learns frequent graph patterns; it also derives $RWK^+$Conv, a GNN layer obtained by unrolling $RWK^+$ computations. Key contributions include structural colors and a diversity regularizer to foster descriptive, non-overlapping hidden graphs, and a principled normalization (StepNorm) to balance step-wise similarities. Extensive experiments demonstrate descriptive learning on pattern mining and superior or competitive performance across supervised graph tasks, including large-scale web data, highlighting practical impact and broader applicability to graph analytics.

Abstract

Graph kernels used to be the dominant approach to feature engineering for structured data, which are superseded by modern GNNs as the former lacks learnability. Recently, a suite of Kernel Convolution Networks (KCNs) successfully revitalized graph kernels by introducing learnability, which convolves input with learnable hidden graphs using a certain graph kernel. The random walk kernel (RWK) has been used as the default kernel in many KCNs, gaining increasing attention. In this paper, we first revisit the RWK and its current usage in KCNs, revealing several shortcomings of the existing designs, and propose an improved graph kernel RWK+, by introducing color-matching random walks and deriving its efficient computation. We then propose RWK+CN, a KCN that uses RWK+ as the core kernel to learn descriptive graph features with an unsupervised objective, which can not be achieved by GNNs. Further, by unrolling RWK+, we discover its connection with a regular GCN layer, and propose a novel GNN layer RWK+Conv. In the first part of experiments, we demonstrate the descriptive learning ability of RWK+CN with the improved random walk kernel RWK+ on unsupervised pattern mining tasks; in the second part, we show the effectiveness of RWK+ for a variety of KCN architectures and supervised graph learning tasks, and demonstrate the expressiveness of RWK+Conv layer, especially on the graph-level tasks. RWK+ and RWK+Conv adapt to various real-world applications, including web applications such as bot detection in a web-scale Twitter social network, and community classification in Reddit social interaction networks.

Figures (6)

• Figure 1: Task 2-1: GED-based evaluation on tail-triangles.
• Figure 2: Task 2-1: GED-based evaluation on rings.
• Figure 3: Runtime of KerGNN+RWK$^{+}$ computed by regular Eqn. \ref{['eq:rw-cm']} vs. efficient Eqn. \ref{['eq:rw-cm-efficient']}, also compared to vanilla RWK.
• Figure 4: Simple Subgraph Matching in Bipartite Graphs
• Figure 5: Simple Subgraph Matching in Triangle Chain

Theorems & Definitions (1)

• Definition 1