Graph Kernel Neural Networks

TL;DR

This work introduces Graph Kernel Neural Networks (GKNN), a fully structural approach that generalizes convolution to graphs through graph kernels and learnable structural masks, avoiding explicit graph embeddings. GKNN defines Graph Kernel Convolution (GKC) layers that compare radius-$r$ subgraphs against learned masks using arbitrary graph kernels, with node features pooled into graph representations for prediction. The model optimizes a cross-entropy loss plus a Jensen–Shannon regularizer over mask distributions, using importance sampling to efficiently estimate kernel expectations and gradients. GKNN demonstrates competitive performance on graph classification and regression benchmarks, offers interpretable learned substructures, and argues for greater expressive power than standard 1-WL-based GNNs, while outlining limitations and avenues for extension.

Abstract

The convolution operator at the core of many modern neural architectures can effectively be seen as performing a dot product between an input matrix and a filter. While this is readily applicable to data such as images, which can be represented as regular grids in the Euclidean space, extending the convolution operator to work on graphs proves more challenging, due to their irregular structure. In this paper, we propose to use graph kernels, i.e. kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain. This allows us to define an entirely structural model that does not require computing the embedding of the input graph. Our architecture allows to plug-in any type of graph kernels and has the added benefit of providing some interpretability in terms of the structural masks that are learned during the training process, similarly to what happens for convolutional masks in traditional convolutional neural networks. We perform an extensive ablation study to investigate the model hyper-parameters' impact and show that our model achieves competitive performance on standard graph classification and regression datasets.

Figures (11)

• Figure 1: The proposed GKNN architecture. The input graph is fed into one or more GKC layers, where subgraphs centered at each node are compared to a series of structural masks through a kernel function. The output is a new set of real-valued feature vectors associated to the graph nodes. We obtain a graph-level feature vector through pooling on the nodes features, which is then fed to an MLP to output the final classification label.
• Figure 2: Example of model training. Top: the matrix of edge observation probabilities associated to the mask $\mathcal{M}_i$ for successive epochs. Bottom: the corresponding mode graphs.
• Figure 3: Assuming the set of masks $M$ spans the space of graphs over $n$ nodes, if the radius of each $N_{\mathcal{G}_1}^r(v)$ is such that it spans the entire graph, the responses on the nodes of $\mathcal{G}_1$ will be identical and will peak for any mask fractionally equivalent to $\mathcal{G}_1$. Here, we use blue and red to highlight the masks with the highest response on $\mathcal{G}_1$ and $\mathcal{G}_2$, respectively.
• Figure 4: $\mathcal{G}_1$ and $\mathcal{G}_2$ cannot be distinguished by the 1-WL test, however the GKNN correctly identifies them as non-isomorphic graphs. If $M$ is the set of all structural masks representing stars up to $n-1$ nodes, no star mask will achieve maximum similarity on the radius 1 subgraphs centered on the nodes of $\mathcal{G}_1$, while the same maximal response will be achieved when considering the radius 1 subgraphs centered on the nodes of $\mathcal{G}_2$.
• Figure 5: Ablation study: bar plots of classification accuracy with standard error. Left to right, top to bottom: number of nodes (i.e., structural mask size), number of structural masks, kernel functions (WL, WL with Optimal Assignment, Graphlet, Propagation, and Pyramid match kernel), subgraph radius, number of GKC layers, and weight of the JSD loss.