The Case for External Graph Sketching

TL;DR

The paper introduces the external semi-streaming model, combining streaming input, sublinear RAM, and disk-based I/O to address practical limitations of graph sketching. It presents a general transformation that converts vertex-based sketches into low-I/O, externally memory-friendly procedures, and proves near-tight I/O lower bounds via a sparse matrix reduction. It then delivers external semi-streaming algorithms for connectivity, hypergraph connectivity, bipartiteness, near-MST weight, k-edge-connectivity, approximate min-cut, cut sparsifiers, and densest subgraph, often matching or surpassing existing external-memory I/O performance while using $O(V ext{polylog}V)$ space. The results show that many sketching techniques can be made practical on modern hardware by leveraging external-memory strategies, and they propose extraction techniques that minimize I/O when recovering multiple components or cuts from sketches. Overall, the work highlights I/O efficiency as a first-class consideration in semi-streaming algorithm design and demonstrates substantial practical potential for graph sketching in large-scale settings.

Abstract

Algorithms in the data stream model use $O(polylog(N))$ space to compute some property of an input of size $N$, and many of these algorithms are implemented and used in practice. However, sketching algorithms in the graph semi-streaming model use $O(V polylog(V))$ space for a $V$-vertex graph, and the fact that implementations of these algorithms are not used in the academic literature or in industrial applications may be because this space requirement is too large for RAM on today's hardware. In this paper we introduce the external semi-streaming model, which addresses the aspects of the semi-streaming model that limit its practical impact. In this model, the input is in the form of a stream and $O(V polylog(V))$ space is available, but most of that space is accessible only via block I/O operations as in the external memory model. The goal in the external semi-streaming model is to simultaneously achieve small space and low I/O cost. We present a general transformation from any vertex-based sketch algorithm to one which has a low sketching cost in the new model. We prove that this automatic transformation is tight or nearly (up to a $O(\log(V))$ factor) tight via an I/O lower bound for the task of sketching the input stream. Using this transformation and other techniques, we present external semi-streaming algorithms for connectivity, bipartiteness testing, $(1+ε)$-approximating MST weight, testing k-edge connectivity, $(1+ε)$-approximating the minimum cut of a graph, computing $ε$-cut sparsifiers, and approximating the density of the densest subgraph. These algorithms all use $O(V poly(\log(V), ε^{-1},k)$ space. For many of these problems, our external semi-streaming algorithms outperform the state of the art algorithms in both the sketching and external-memory models.

Theorems & Definitions (20)

• Definition 1: Vertex-based sketchGuhaMT15
• Theorem 4.1
• Definition 2
• Lemma 4.1
• Lemma 4.2
• Corollary 4.1
• Theorem 5.1
• Theorem 5.2
• Lemma 5.1
• Corollary 5.1