Computationally-efficient Graph Modeling with Refined Graph Random Features

TL;DR

This paper tackles the computational bottleneck of kernel methods on graphs by introducing refined Graph Random Features (GRFs++), which use walk-stitching to emulate longer random walks via multiple short, parallel walks while preserving unbiasedness. GRFs++ also generalize walk termination beyond Bernoulli schemes, enabling more accurate kernel approximations without extra cost. The authors provide a theoretical framework showing unbiasedness and concentration, plus practical strategies (three deployment options, parallelization, and walk reuse) to achieve speedups. Empirically, GRFs++ achieve tighter kernel approximations than regular GRFs across diverse graphs and improve performance on graph classification, clustering, and mesh normal prediction, highlighting significant practical impact for scalable graph kernels.

Abstract

We propose refined GRFs (GRFs++), a new class of Graph Random Features (GRFs) for efficient and accurate computations involving kernels defined on the nodes of a graph. GRFs++ resolve some of the long-standing limitations of regular GRFs, including difficulty modeling relationships between more distant nodes. They reduce dependence on sampling long graph random walks via a novel walk-stitching technique, concatenating several shorter walks without breaking unbiasedness. By applying these techniques, GRFs++ inherit the approximation quality provided by longer walks but with greater efficiency, trading sequential, inefficient sampling of a long walk for parallel computation of short walks and matrix-matrix multiplication. Furthermore, GRFs++ extend the simplistic GRFs walk termination mechanism (Bernoulli schemes with fixed halting probabilities) to a broader class of strategies, applying general distributions on the walks' lengths. This improves the approximation accuracy of graph kernels, without incurring extra computational cost. We provide empirical evaluations to showcase all our claims and complement our results with theoretical analysis.

Figures (8)

• Figure 1: Pictorial description of the walk-stitching technique. Each rectangular block corresponds to a random walk and red nodes depict vertices where walks meet. The blue nodes are the communicating ones. The thickness of the arrow, depicting a transition from step $t$ to step $t+1$, indicates the probability that such a transition will occur (a walk can terminate earlier). Top: In regular GRFs, two graph vertices communicate via intersecting walks, originating at each vertex. As the nodes become more distant, the probability that such two walks will be constructed decreases. Bottom: In GRFs++, two nodes communicate with each other less directly, via proxies (the middle blue node in the picture) and much shorter walks, with lengths that have much higher probability of being realized. The communication is established by stitching several small walks.
• Figure 2: Comparison of different GRF methods for the diffusion kernel estimation. The approximation error (y-axis) improves with the number of walks $m$ (x-axis) and GRF++ provides a sharper estimate than the previous GRF mechanism. The experiment is repeated $s=10$ times.
• Figure 3: Our novel halting policy based on Poisson distribution provides additional gains over the GRF mechanisms. We run the experiment $s=10$ times on different graphs of varying sizes.
• Figure 4: Using the exact same walk as the baseline GRF, repeated multiple times, pinpoints the effectiveness of the walk-stitching algorithm, showing additional computational gains.
• Figure 5: Speed comparison for various GRF-methods: regular GRFs and GRFs++ with different degrees.

Theorems & Definitions (9)

• Lemma 2.1: Unbiased walk-stitching and higher-level convolutions
• Lemma 3.1: Unbiased walk-stitching, higher-order convolutions & general termination
• Lemma 3.2: MSE of the approximation via GRFs++ with $l=2$
• Theorem 3.3
• Remark 3.4
• proof
• Lemma A.1: MSE of the GRFs++ based graph estimator with GRFs++ degree $l=2$
• proof
• proof