Efficient Learning on Large Graphs using a Densifying Regularity Lemma

TL;DR

The paper addresses scalability of graph learning on very large and/or sparse directed graphs by introducing Intersecting Blocks Graphs (IBG) as a low-rank, directed-structure representation. Central to the approach is a semi-constructive, densifying weak regularity lemma that guarantees approximating any graph by a dense IBG with rank depending only on the target accuracy, not graph size or sparsity. The authors design IBG-NNs, neural networks that operate directly on the IBG representation, achieving linear-in-N time/space per layer and enabling efficient node classification, spatio-temporal analysis, and knowledge graph completion. Empirical results show state-of-the-art performance on directed benchmarks and large-scale graphs, while preserving substantial computational and memory advantages over traditional MPNNs and prior ICG-based methods. The work combines theoretical densification with practical subgraph SGD strategies and a flexible architecture to deliver scalable, high-performance graph learning for real-world large-scale applications.

Abstract

Learning on large graphs presents significant challenges, with traditional Message Passing Neural Networks suffering from computational and memory costs scaling linearly with the number of edges. We introduce the Intersecting Block Graph (IBG), a low-rank factorization of large directed graphs based on combinations of intersecting bipartite components, each consisting of a pair of communities, for source and target nodes. By giving less weight to non-edges, we show how to efficiently approximate any graph, sparse or dense, by a dense IBG. Specifically, we prove a constructive version of the weak regularity lemma, showing that for any chosen accuracy, every graph, regardless of its size or sparsity, can be approximated by a dense IBG whose rank depends only on the accuracy. This dependence of the rank solely on the accuracy, and not on the sparsity level, is in contrast to previous forms of the weak regularity lemma. We present a graph neural network architecture operating on the IBG representation of the graph and demonstrating competitive performance on node classification, spatio-temporal graph analysis, and knowledge graph completion, while having memory and computational complexity linear in the number of nodes rather than edges.

Figures (9)

• Figure 1: (a) Directed graph sampled from a stochastic block model (SBM); (b) The approximating 3-IBG, where source and target community pairs are represented by the same color; (c) Adjacency matrix of a graph sampled from the same SBM, with the 3-IBG overlaid on the adjacency matrix.
• Figure 2: Runtime of K-IBG-NN as a function of DirGNN forward pass duration on sparse $\mathrm{ER}(n, p=25/n)$ graphs (left) and dense $\mathrm{ER}(n, p=0.5)$ graphs (right) for K=10, 100.
• Figure 3: Accuracy of IBG-NNs and ICG-NNs on the Squirrel (left) and Chameleon (right) datasets as a function of the number of communities.
• Figure 4: Densifying cut similarity as a function of the weighted Frobenius norm on the Squirrel (left) and Chameleon (right) datasets. The number of communities is specified above every point.
• Figure 5: IBG-NN accuracy over the sparse dataset Arxiv-Year as a function of $\Gamma$. The dotted line is IBG-NN accuracy when using standard uniformly weighted loss for the IBG approximation. The rightmost point is the value of $\Gamma$ which results in a uniform cut norm under the densifying loss.

Theorems & Definitions (34)

• Definition 3.1
• Definition 3.2
• Definition 4.1
• Definition 4.2
• Theorem 4.1
• Proposition 4.2
• Theorem B.1: Weak Regularity Lemma frieze1999quick
• Remark C.1
• Definition C.1
• Definition D.1