Reservoir Sampling over Joins

TL;DR

The paper tackles the challenge of maintaining a uniform random sample of join results in a streaming setting without materializing the full join. It introduces a generalized reservoir sampling algorithm that handles predicates and a dynamic index for acyclic joins, enabling near-linear total time $O\left(N \log N + k \log N \log \frac{N}{k}\right)$ and amortized $O(\log N)$ update costs. It then extends this framework to acyclic and cyclic joins (via generalized hypertree decompositions), achieving provable efficiency with space $O(N^{w})$ and time $O\left(N^{w} \log N + k \log N \log \frac{N}{k}\right)$ where $w$ is the fractional hypertree width. Extensive experiments on graph and relational workloads demonstrate substantial performance gains over prior state-of-the-art methods, validating both the theoretical guarantees and practical applicability for large-scale streaming analytics.

Abstract

Sampling over joins is a fundamental task in large-scale data analytics. Instead of computing the full join results, which could be massive, a uniform sample of the join results would suffice for many purposes, such as answering analytical queries or training machine learning models. In this paper, we study the problem of how to maintain a random sample over joins while the tuples are streaming in. Without the join, this problem can be solved by some simple and classical reservoir sampling algorithms. However, the join operator makes the problem significantly harder, as the join size can be polynomially larger than the input. We present a new algorithm for this problem that achieves a near-linear complexity. The key technical components are a generalized reservoir sampling algorithm that supports a predicate, and a dynamic index for sampling over joins. We also conduct extensive experiments on both graph and relational data over various join queries, and the experimental results demonstrate significant performance improvement over the state of the art.

Figures (13)

• Figure 1: An illustration of reservoir sampling over join.
• Figure 2: An illustration of index structure built for line-3 join $R_1(X,Y) \Join R_2(Y,Z)\Join R_3(Z,W)$.
• Figure 3: An illustration of grouping optimization
• Figure 4: The dumbbell join $\mathcal{Q} = R_1(x_1,x_2)\Join R_2(x_1,x_3) \Join R_3(x_2, x_3) \Join R_4(x_5,x_6) \Join R_5(x_4,x_5) \Join R_6(x_4,x_6) \Join R_7(x_3,x_4)$ with GHD illustrated as the red circle. It has fractional hypertree width $\textsf{w}(\mathcal{Q})=1.5$ since the triangle join $R_1(x_1,x_2)\Join R_2(x_1,x_3) \Join R_3(x_2, x_3)$ and $R_5(x_4,x_5) \Join R_6(x_4,x_6) \Join R_7(x_3,x_4)$ have the fractional edge covering number $\rho^*=1.5$.
• Figure 5: Running time over different join queries

Theorems & Definitions (17)

• theorem 1
• theorem 2
• theorem 3
• definition 1: Dense stream
• corollary 1
• lemma 1
• lemma 2
• lemma 3
• theorem 4
• definition 2: Acyclic Join beeri1983desirability