Faster Local Solvers for Graph Diffusion Equations

TL;DR

The paper tackles the high cost of computing graph diffusion equations on large graphs by exploiting the strong localization of diffusion vectors. It introduces a general local diffusion process framework and three solver families (LocalSOR, LocalGD, LocalCH) that localize standard iterative methods, achieving sublinear runtime under suitable conditions and being highly GPU-friendly. The authors demonstrate up to 100× speedups across 18 real-world graphs for PPR, Katz, and HK; they also extend the approach to dynamic GDEs and fast GNN propagation, notably enhancing InstantGNN-style training. While APPR is shown to be a special case of the framework, the paper also discusses limitations and open questions, such as proving accelerated bounds for the proposed solvers and extending the approach to broader GDEs.

Abstract

Efficient computation of graph diffusion equations (GDEs), such as Personalized PageRank, Katz centrality, and the Heat kernel, is crucial for clustering, training neural networks, and many other graph-related problems. Standard iterative methods require accessing the whole graph per iteration, making them time-consuming for large-scale graphs. While existing local solvers approximate diffusion vectors through heuristic local updates, they often operate sequentially and are typically designed for specific diffusion types, limiting their applicability. Given that diffusion vectors are highly localizable, as measured by the participation ratio, this paper introduces a novel framework for approximately solving GDEs using a local diffusion process. This framework reveals the suboptimality of existing local solvers. Furthermore, our approach effectively localizes standard iterative solvers by designing simple and provably sublinear time algorithms. These new local solvers are highly parallelizable, making them well-suited for implementation on GPUs. We demonstrate the effectiveness of our framework in quickly obtaining approximate diffusion vectors, achieving up to a hundred-fold speed improvement, and its applicability to large-scale dynamic graphs. Our framework could also facilitate more efficient local message-passing mechanisms for GNNs.

Figures (35)

• Figure 1: The maximal participation ratio $p({\bm{f}})=(\sum_{i=1}^n |f_i|^2)^2 / (n \sum_{i=1}^n |f_i|^4)$ of example diffusion vectors ${\bm{f}}$ over 18 graphs, ordered from small (cora) to large (ogbn-papers100M). The ratio $p({\bm{f}})$ is normalized by the number of nodes $n$.
• Figure 2: The first row illustrates the local diffusion process of APPR andersen2006local over a toy network topology adopted from hamilton2017inductive. It uses $T_{\operatorname{APPR}}=6$ local iterations with ${\mathcal{T}}_{\operatorname{APPR}} = 42$ operations and additive error $\approx 0.29241$. The second row shows the process of LocalSOR ($\omega = 1.19 \approx \omega^*$). It uses $T_{\operatorname{LocalSOR}} = 5$ local iterations with ${\mathcal{T}}_{\operatorname{LocalSOR}} = 28$ operations and additive error $\approx 0.21479$. LocalSOR uses fewer local iterations, costs less total active volume, and obtains better approximate solutions. We choose the source node $s=0$ with $\epsilon = 0.02$ and $\alpha = 0.25$.
• Figure 3: Parameter tuning of $\omega$.
• Figure 4: Number of operations required for four representative methods and their localized counterparts over 18 graphs. The graph index is sorted according to the performance of LocalGS.
• Figure 5: The number of operations as a function of ${\epsilon}$ for comparing LocalSOR and LocalGS.

Theorems & Definitions (16)

• Definition 3.1: Local diffusion process
• Theorem 3.2: Properties of local diffusion process via LocalSOR
• Theorem 3.3: Sublinear runtime bound of LocalSOR for PPR
• Corollary 3.4: Runtime bound of LocalSOR for Katz
• Theorem 3.5: Properties of local diffusion process via LocalGD
• Corollary 3.6: Convergence of LocalGD for PPR and Katz
• Remark 3.7
• Proposition B.1: avrachenkov2013choicelofgren2015bidirectional
• proof
• proof