Weighted Minwise Hashing Beats Linear Sketching for Inner Product Estimation

TL;DR

This work tackles the challenge of estimating inner products between high-dimensional vectors under compact sketches. It introduces Weighted MinHash as a non-linear sketching method that extends MinHash to weighted, real-valued vectors, providing a worst-case bound that matches linear sketches for dense vectors and improves when overlap between supports is small. The method expands vectors with weights, applies Weighted MinHash, and uses an estimator with $m = O(\log(1/\delta)/\epsilon^2)$ to guarantee accurate approximations with high probability, even for sparse data. Empirical results on synthetic and real datasets show WMH outperforming traditional linear sketches and unweighted hashing-based sketches in sparse regimes, with competitive performance when overlap is large, highlighting its practical impact for dataset search, join-size estimation, and related statistics on unjoined tables.

Abstract

We present a new approach for computing compact sketches that can be used to approximate the inner product between pairs of high-dimensional vectors. Based on the Weighted MinHash algorithm, our approach admits strong accuracy guarantees that improve on the guarantees of popular linear sketching approaches for inner product estimation, such as CountSketch and Johnson-Lindenstrauss projection. Specifically, while our method admits guarantees that exactly match linear sketching for dense vectors, it yields significantly lower error for sparse vectors with limited overlap between non-zero entries. Such vectors arise in many applications involving sparse data. They are also important in increasingly popular dataset search applications, where inner product sketches are used to estimate data covariance, conditional means, and other quantities involving columns in unjoined tables. We complement our theoretical results by showing that our approach empirically outperforms existing linear sketches and unweighted hashing-based sketches for sparse vectors.

Figures (5)

• Figure 1: The table $\mathcal{T}_{A \bowtie B}$ is the output of a one-to-one join between the tables $\mathcal{T}_A$ with $\mathcal{T}_B$. We are interested in approximating post-join statistics (e.g., join size, sums, means, and covariances) of the table $\mathcal{T}_{A \bowtie B}$ using only inner products.
• Figure 2: Vector representation of tables $\mathcal{T}_A$ with $\mathcal{T}_B$ from Figure \ref{['fig:example-tables']}. The vector $\mathbf{x}^{\mathbbm{1}[K_A]}$ (resp. $\mathbf{x}^{\mathbbm{1}[K_B]}$) is the vector representation for the join key $K_A$ (resp. $K_B$) and $\mathbf{x}^{V_A}$ (resp. $\mathbf{x}^{V_B}$) is the vector representation for the column $V_A$ (resp. $V_B$). Bold numbers are entries included in the join result from $\mathcal{T}_{A \bowtie B}$.
• Figure 3: Inner product estimation (synthetic data).
• Figure 4: Inner product estimation (World Bank data). Different shades of blue highlight combinations for which WMH outperforms the other methods.
• Figure 5: Text similarity estimation (20 Newsgroups dataset). Note that in the left plot, the lines for MH, WMH, and KMV all lie essentially on top of one another.

Theorems & Definitions (10)

• Theorem 2: Main Result
• Theorem 4: Intermediate Result: Inner Product Sketching with Unweighted MinHash
• Lemma 1: Union Size Estimator BlumHopcroftKannan:2020
• proof : Proof of \ref{['thm:bounded']}
• proof : Proof of \ref{['fact:mhash_sketchWeighted']}
• Lemma 2
• proof
• Lemma 3: Rounding
• proof
• proof : Proof of \ref{['thm:main']}