Heating Up Quasi-Monte Carlo Graph Random Features: A Diffusion Kernel Perspective

TL;DR

It is asserted that q-GRFs achieve lower variance estimators of the Diffusion (or Heat) kernel on Ladder graphs, and the number of rungs on the Ladder graphs impacts the algorithm's performance; further theoretical results supporting this assertion are forthcoming.

Abstract

We build upon a recently introduced class of quasi-graph random features (q-GRFs), which have demonstrated the ability to yield lower variance estimators of the 2-regularized Laplacian kernel (Choromanski 2023). Our research investigates whether similar results can be achieved with alternative kernel functions, specifically the Diffusion (or Heat), Matérn, and Inverse Cosine kernels. We find that the Diffusion kernel performs most similarly to the 2-regularized Laplacian, and we further explore graph types that benefit from the previously established antithetic termination procedure. Specifically, we explore Erdős-Rényi and Barabási-Albert random graph models, Binary Trees, and Ladder graphs, with the goal of identifying combinations of specific kernel and graph type that benefit from antithetic termination. We assert that q-GRFs achieve lower variance estimators of the Diffusion (or Heat) kernel on Ladder graphs. However, the number of rungs on the Ladder graphs impacts the algorithm's performance; further theoretical results supporting our experimentation are forthcoming. This work builds upon some of the earliest Quasi-Monte Carlo methods for kernels defined on combinatorial objects, paving the way for kernel-based learning algorithms and future real-world applications in various domains.

Figures (16)

• Figure 1: Relative Frobenius norm error of estimators of the Diffusion (or heat) kernel with $t = 0.5$ using general GRFs (red circles) and q-GRFs (green crosses). Lower is better. $N$ gives the number of nodes and $p$ is the edge-generation probability for the Erdös-Rényi graphs. One standard deviation is shaded, but in some of the graphs it is too small to easily see. The novel q-GRFs algorithm performed better on all eight graphs, mirroring the behavior of the previosuly studied 2-regularized Laplacian kernel reid_quasi-monte_2023. For the sake of demonstrating long-term behavior, we visualize 50 random walks where clearly q-QRFs yield a lower variance estimator for the Diffusion kernel as number of random walks increases.
• Figure 2: Relative Frobenius norm error of estimators of the 2-regularized Laplacian kernel using GRFs (red circles) and q-GRFs (green crosses). Lower is better. $N$ gives the number of nodes and $p$ is the edge-generation probability for the Erdös-Rényi graphs. One standard deviation is shaded, but it is too small to easily see. These graphs are a direct result of a previous study by Choromanski, Reid, and Weller that introduced Quasi-Monte Carlo antithetic termination on random walks. These results serve as a baseline for comparing alternative kernel functions.
• Figure 3: Relative Frobenius norm error of estimators of the Matérn kernel with smoothness parameter $\nu = 2.5$ and length scale parameter $l = 1$ using GRFs (red circles) and q-GRFs (green crosses). Lower is better. $N$ gives the number of nodes and $p$ is the edge-generation probability for the Erdös-Rényi graphs. One standard deviation is shaded, but in some of the graphs it is too small to easily see. Q-GRFs performed worse or, at best, the same as regular g-GRFs.
• Figure 4: Relative Frobenius norm error of estimators of the inverse cosine kernel using GRFs (red circles) and q-GRFs (green crosses). Lower is better. $N$ gives the number of nodes and $p$ is the edge-generation probability for the Erdös-Rényi graphs. One standard deviation is shaded, but in some of the graphs it is too small to easily see. The q-GRFs algorithm performed best on Erdös-Rényi, Binary Tree, Ladder, and Dolphins graphs.
• Figure 5: 50 Erdős-Rényi graphs with a spin parameter set to 20. Q-GRF's yield lower variance estimators of the Diffusion kernel 26% of the time.