RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search

TL;DR

RaBitQ tackles the lack of theoretical guarantees in product quantization for approximate nearest neighbor search by introducing a randomized, bi-valued codebook on the unit hypersphere and an unbiased estimator for inner products. The method combines a carefully constructed codebook with a probabilistic distance estimator that has a sharp $O(1/\sqrt{D})$ error bound and efficient single- and batch-distance computations via bitwise operations and SIMD-friendly pipelines. Theoretical results are complemented by practical in-memory ANN integration using IVF and a re-ranking strategy guided by the bound, eliminating parameter tuning. Empirical evaluations on real datasets demonstrate superior time-accuracy trade-offs, robustness across datasets, and alignment with theory, offering a scalable, provably accurate alternative to PQ-family methods.

Abstract

Searching for approximate nearest neighbors (ANN) in the high-dimensional Euclidean space is a pivotal problem. Recently, with the help of fast SIMD-based implementations, Product Quantization (PQ) and its variants can often efficiently and accurately estimate the distances between the vectors and have achieved great success in the in-memory ANN search. Despite their empirical success, we note that these methods do not have a theoretical error bound and are observed to fail disastrously on some real-world datasets. Motivated by this, we propose a new randomized quantization method named RaBitQ, which quantizes $D$-dimensional vectors into $D$-bit strings. RaBitQ guarantees a sharp theoretical error bound and provides good empirical accuracy at the same time. In addition, we introduce efficient implementations of RaBitQ, supporting to estimate the distances with bitwise operations or SIMD-based operations. Extensive experiments on real-world datasets confirm that (1) our method outperforms PQ and its variants in terms of accuracy-efficiency trade-off by a clear margin and (2) its empirical performance is well-aligned with our theoretical analysis.

Figures (11)

• Figure 1: Geometric Relationship among the Vectors.
• Figure 2: Bitwise Decomposition of $\mathbf{\bar{q}}_u$.
• Figure 3: Time-Accuracy Trade-Off for Distance Approximation. For baseline methods, (1) "x4fs-batch" means that the SIMD-based fast implementation is adopted (where 4 bits encode a quantized code and approximate distances for a batch of 32 data vectors are computed each time), and (2) "x8-single" means that 8 bits encode a quantized code and the approximate distance of one data vector is computed each time. In addition, the results of LSQx8-single are omitted since it, with the implementation from Faiss, has the time cost significantly larger than others.
• Figure 4: Time-Accuracy Trade-Off for ANN Search. The parameter $rerank$ represents the number of candidates for re-ranking.
• Figure 5: Verification Study on $\epsilon_0$.

Theorems & Definitions (13)

• lemma 1: Geometric Relationship
• theorem 1: Estimator
• theorem 2
• proof
• lemma 2
• proof
• lemma 3
• proof
• lemma 4: Distribution
• proof