Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Search Neural Networks for Efficient and Expressive Graph Learning

Michael Ito, Danai Koutra, Jenna Wiens

TL;DR

RWNNs are powerful but struggle to capture global structure under partial coverage. We introduce RSNNs, which replace walks with random DFS-based searches to guarantee full node coverage and, in sparse graphs, full edge coverage with only $O( abla abla |V|)$ searches, enabling universal approximation when paired with expressive sequence models and preserving isomorphism invariance in expectation. Theoretical guarantees are matched by empirical results on sparse molecular and protein benchmarks, where RSNNs consistently outperform RWNN baselines while using up to 16x fewer samples. This work provides a principled, theory-backed pathway to efficient and expressive graph learning on sparse graphs.

Abstract

Random walk neural networks (RWNNs) have emerged as a promising approach for graph representation learning, leveraging recent advances in sequence models to process random walks. However, under realistic sampling constraints, RWNNs often fail to capture global structure even in small graphs due to incomplete node and edge coverage, limiting their expressivity. To address this, we propose \textit{random search neural networks} (RSNNs), which operate on random searches, each of which guarantees full node coverage. Theoretically, we demonstrate that in sparse graphs, only $O(\log |V|)$ searches are needed to achieve full edge coverage, substantially reducing sampling complexity compared to the $O(|V|)$ walks required by RWNNs (assuming walk lengths scale with graph size). Furthermore, when paired with universal sequence models, RSNNs are universal approximators. We lastly show RSNNs are probabilistically invariant to graph isomorphisms, ensuring their expectation is an isomorphism-invariant graph function. Empirically, RSNNs consistently outperform RWNNs on molecular and protein benchmarks, achieving comparable or superior performance with up to 16$\times$ fewer sampled sequences. Our work bridges theoretical and practical advances in random walk based approaches, offering an efficient and expressive framework for learning on sparse graphs.

Random Search Neural Networks for Efficient and Expressive Graph Learning

TL;DR

RWNNs are powerful but struggle to capture global structure under partial coverage. We introduce RSNNs, which replace walks with random DFS-based searches to guarantee full node coverage and, in sparse graphs, full edge coverage with only searches, enabling universal approximation when paired with expressive sequence models and preserving isomorphism invariance in expectation. Theoretical guarantees are matched by empirical results on sparse molecular and protein benchmarks, where RSNNs consistently outperform RWNN baselines while using up to 16x fewer samples. This work provides a principled, theory-backed pathway to efficient and expressive graph learning on sparse graphs.

Abstract

Random walk neural networks (RWNNs) have emerged as a promising approach for graph representation learning, leveraging recent advances in sequence models to process random walks. However, under realistic sampling constraints, RWNNs often fail to capture global structure even in small graphs due to incomplete node and edge coverage, limiting their expressivity. To address this, we propose \textit{random search neural networks} (RSNNs), which operate on random searches, each of which guarantees full node coverage. Theoretically, we demonstrate that in sparse graphs, only searches are needed to achieve full edge coverage, substantially reducing sampling complexity compared to the walks required by RWNNs (assuming walk lengths scale with graph size). Furthermore, when paired with universal sequence models, RSNNs are universal approximators. We lastly show RSNNs are probabilistically invariant to graph isomorphisms, ensuring their expectation is an isomorphism-invariant graph function. Empirically, RSNNs consistently outperform RWNNs on molecular and protein benchmarks, achieving comparable or superior performance with up to 16 fewer sampled sequences. Our work bridges theoretical and practical advances in random walk based approaches, offering an efficient and expressive framework for learning on sparse graphs.
Paper Structure (51 sections, 19 theorems, 56 equations, 4 figures, 4 tables, 2 algorithms)

This paper contains 51 sections, 19 theorems, 56 equations, 4 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. Notation and Random Walks on Graphs
  4. Message-passing Neural Networks and GNN Expressivity
  5. Random Walk Neural Networks
  6. Random Walk Cover Times
  7. Expressive Power of Random Walk Neural Networks
  8. The Role of Coverage: RWNNs vs. MPNNs
  9. Random Search Neural Networks (RSNNs)
  10. Search via Random DFS
  11. From Efficient Graph Coverage to Universal Approximation in RSNNs
  12. From Expressivity to Invariance: Isomorphism Invariance of RSNNs
  13. Learning the invariance.
  14. Inference.
  15. Runtime Complexity
  16. ...and 36 more sections

Key Result

Theorem 3.1

Let $G$ be a graph. Let $f_{\mathrm{RWNN}}^{\mathrm{FC}}$ denote an RWNN with injective $f_\mathrm{seq}$ and $f_\mathrm{agg}$ with no additional positional encodings, applied to the complete multiset of walks $\mathcal{W}_{\le \ell}(G)$ with lengths up to $\ell=C_E(G)$, the edge cover time of $G$. L Hence, $f_{\mathrm{RWNN}}^{\mathrm{FC}} \simeq f_{\mathrm{MPNN}}$ (i.e., $f_{\mathrm{RWNN}}^{\mathr

Figures (4)

  • Figure 1: RWNN and RSNN coverage differences. Random walks miss critical structure under realistic sampling constraints, wheras each individual search only misses single edges in cycles, enabling complete reconstruction across logarithmic sampling in $\lvert V \rvert$ on sparse graphs.
  • Figure 2: Overview of an RSNN layer. Starting from an input graph, $m$ random depth-first searches are extracted and encoded via $f_{\text{emb}}$. Additional positional encodings indicate discontinuities in the sequence (e.g., -$\cdot$-in search 1). These sequences are processed by a sequence model $f_{\text{seq}}$, and final node representations are aggregated across sequences using $f_{\text{agg}}$. We highlight in blue the flow of a selected node representation (shown as ) as it is tracked through each stage of the RSNN layer.
  • Figure 3: Coverage vs. performance across benchmarks. RSNNs achieve higher coverage and performance at low sample sizes, while CRAWL only approaches RSNN coverage and performance at $m=16$, highlighting a strong correlation between coverage and performance.
  • Figure 4: Training runtime (in seconds) of RSNN and CRAWL over 25 epochs on SIDER, BBBP, TOXCAST as a function of the number of samples $m$. Error bars represent standard deviation across 5 runs. At low sample counts, RSNNs exhibit comparable runtime to RWNNs; as $m$ increases, RSNNs become faster despite longer sequence lengths. We hypothesize this is due to random walks repeatedly visiting high-degree nodes, incurring more computation per step, whereas DFS-based searches visit each node exactly once.

Theorems & Definitions (33)

  • Theorem 3.1: RWNN-MPNN Equivalence Under Full Coverage (FC)
  • Corollary 3.2: RWNNs Under Partial Coverage (PC)
  • Lemma 4.1: Logarithmic Sampling Yields Full Edge Coverage
  • Theorem 4.2: Universal Approximation by RSNNs on Sparse Graphs with Bounded Degree
  • Theorem 4.3: Probabilistic Isomorphism-Invariance of RSNN
  • Corollary 4.4: SGD converges to the invariant objective
  • Definition A.1: 1-WL color refinement
  • Lemma A.2: MPNN vs. 1-WL Expressivity xu2018howmorris2019weisfeiler
  • Definition A.3: WWL at length $\ell$
  • Lemma A.4: Monotonicity in $t$, $\ell$, and $\pi_0$
  • ...and 23 more