Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster Graph Embeddings via Coarsening

Matthew Fahrbach, Gramoz Goranci, Richard Peng, Sushant Sachdeva, Chi Wang

TL;DR

This work tackles the scalability problem of graph embeddings on large graphs by introducing two vertex-sparsification methods based on Schur complements. The SchurComplement and RandomContraction algorithms reduce the graph while exactly or in expectation preserving the embedding structure for a set of terminal vertices, enabling near-linear-time preprocessing. The authors prove that the embedding matrices, particularly under NetMF, are preserved on the terminals as the walk length grows, and demonstrate substantial speedups and competitive accuracy on multi-label vertex classification and link prediction benchmarks. The results show practical benefits for large-scale networks, making high-quality embeddings feasible for subsets of nodes without sacrificing predictive performance.

Abstract

Graph embeddings are a ubiquitous tool for machine learning tasks, such as node classification and link prediction, on graph-structured data. However, computing the embeddings for large-scale graphs is prohibitively inefficient even if we are interested only in a small subset of relevant vertices. To address this, we present an efficient graph coarsening approach, based on Schur complements, for computing the embedding of the relevant vertices. We prove that these embeddings are preserved exactly by the Schur complement graph that is obtained via Gaussian elimination on the non-relevant vertices. As computing Schur complements is expensive, we give a nearly-linear time algorithm that generates a coarsened graph on the relevant vertices that provably matches the Schur complement in expectation in each iteration. Our experiments involving prediction tasks on graphs demonstrate that computing embeddings on the coarsened graph, rather than the entire graph, leads to significant time savings without sacrificing accuracy.

Faster Graph Embeddings via Coarsening

TL;DR

This work tackles the scalability problem of graph embeddings on large graphs by introducing two vertex-sparsification methods based on Schur complements. The SchurComplement and RandomContraction algorithms reduce the graph while exactly or in expectation preserving the embedding structure for a set of terminal vertices, enabling near-linear-time preprocessing. The authors prove that the embedding matrices, particularly under NetMF, are preserved on the terminals as the walk length grows, and demonstrate substantial speedups and competitive accuracy on multi-label vertex classification and link prediction benchmarks. The results show practical benefits for large-scale networks, making high-quality embeddings feasible for subsets of nodes without sacrificing predictive performance.

Abstract

Graph embeddings are a ubiquitous tool for machine learning tasks, such as node classification and link prediction, on graph-structured data. However, computing the embeddings for large-scale graphs is prohibitively inefficient even if we are interested only in a small subset of relevant vertices. To address this, we present an efficient graph coarsening approach, based on Schur complements, for computing the embedding of the relevant vertices. We prove that these embeddings are preserved exactly by the Schur complement graph that is obtained via Gaussian elimination on the non-relevant vertices. As computing Schur complements is expensive, we give a nearly-linear time algorithm that generates a coarsened graph on the relevant vertices that provably matches the Schur complement in expectation in each iteration. Our experiments involving prediction tasks on graphs demonstrate that computing embeddings on the coarsened graph, rather than the entire graph, leads to significant time savings without sacrificing accuracy.

Paper Structure

This paper contains 18 sections, 4 theorems, 21 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Other Related Works
  4. Preliminaries
  5. Graph Coarsening Algorithms
  6. Guarantees for Graph Embeddings
  7. Experiments
  8. Multi-label Vertex Classification
  9. Datasets.
  10. Evaluation Methods.
  11. Results for LINE (NetMF with $W=1$).
  12. Results for DeepWalk (NetMF with $W \ge 2$).
  13. Link Prediction
  14. Datasets.
  15. Evaluation Methods.
  16. ...and 3 more sections

Key Result

Theorem 3.2

Given a graph $G=(V,E,w)$ with $m$ edges and terminals $T \subseteq V$, the algorithm RandomContraction produces a graph $H$ with $O(m)$ edges that contains the terminals $T$ in $O(m \log n)$ time. Moreover, $H$ preserves $\mathbf{SC}\xspace(\mathbf{M},T)$ in expectation in each iteration.

Figures (2)

  • Figure 1: Accuracy of sparsified LINE embeddings. The $x$-axis denotes the training ratio $(\%)$ and the $y$-axis in the top and bottom row denotes the mean Micro-F1 and Macro-F1 scores, respectively.
  • Figure 2: Accuracy of sparsified DeepWalk embeddings. The $x$-axis denotes the training ratio $(\%)$ and the $y$-axis in the top and bottom row denotes the mean Micro-F1 and Macro-F1 scores, respectively.

Theorems & Definitions (10)

  • Definition 3.1: Random Contraction
  • Theorem 3.2
  • Lemma 3.3
  • proof
  • proof : Proof of Theorem \ref{['lem:randomContract']}
  • Definition 4.1: NetMFSC
  • Theorem 4.2
  • proof : Proof of Theorem \ref{['lem:SCGuarantees']}
  • Lemma A.1
  • proof