Gaussian process regression with Sliced Wasserstein Weisfeiler-Lehman graph kernels

TL;DR

This work focuses on Gaussian process regression, for which it is introduced the Sliced Wasserstein Weisfeiler-Lehman (SWWL) graph kernel, which enjoys positive definiteness and a drastic complexity reduction, which makes it possible to process datasets that were previously impossible to handle.

Abstract

Supervised learning has recently garnered significant attention in the field of computational physics due to its ability to effectively extract complex patterns for tasks like solving partial differential equations, or predicting material properties. Traditionally, such datasets consist of inputs given as meshes with a large number of nodes representing the problem geometry (seen as graphs), and corresponding outputs obtained with a numerical solver. This means the supervised learning model must be able to handle large and sparse graphs with continuous node attributes. In this work, we focus on Gaussian process regression, for which we introduce the Sliced Wasserstein Weisfeiler-Lehman (SWWL) graph kernel. In contrast to existing graph kernels, the proposed SWWL kernel enjoys positive definiteness and a drastic complexity reduction, which makes it possible to process datasets that were previously impossible to handle. The new kernel is first validated on graph classification for molecular datasets, where the input graphs have a few tens of nodes. The efficiency of the SWWL kernel is then illustrated on graph regression in computational fluid dynamics and solid mechanics, where the input graphs are made up of tens of thousands of nodes.

Figures (6)

• Figure 1: SWWL kernel. Step 1: Graph embedding. Step 2: Projected quantile embeddings with only a few quantiles kept. Step 3: Euclidean distances between embeddings.
• Figure 2: Illustration of Gaussian process for graph inputs. Left: samples from the prior distribution. Right: samples from the posterior distribution after conditioning on observations (input points are graphs here).
• Figure 3: Top row: meshes from the Rotor37, Tensile2d and AirfRANS datasets (from left to right). Bottom row: coarsened versions of the meshes.
• Figure 4: Impact of number of projections and quantiles on RMSE for GP regression for Rotor37.
• Figure 5: Distance between embeddings of graphs seen as distributions. The black line represents operations that can be performed separately on each input $G_1, \ldots, G_N$. Step 1: graph embedding. Step 2: distance between distributions.

Theorems & Definitions (10)

• Definition 1: SWWL kernel
• Definition 2: Continuous WL embeddings
• Definition 3: Wasserstein distance
• Definition 4: Sliced Wasserstein distance
• Definition 5: Hilbertian (pseudo)-distance hilbertian
• proof
• proof
• Definition 6: Push-forward operator
• Definition 7: 1-dimensional Wasserstein distance
• Definition 8