Efficient and Effective Retrieval of Dense-Sparse Hybrid Vectors using Graph-based Approximate Nearest Neighbor Search

TL;DR

This work proposes a distribution alignment method to improve accuracy, which pre-samples dense and sparse vectors to analyze their distance distribution statistic, resulting in a 1% increase in accuracy, and designs an adaptive two-stage computation strategy that initially computes dense distances only and later computes hybrid distances.

Abstract

ANNS for embedded vector representations of texts is commonly used in information retrieval, with two important information representations being sparse and dense vectors. While it has been shown that combining these representations improves accuracy, the current method of conducting sparse and dense vector searches separately suffers from low scalability and high system complexity. Alternatively, building a unified index faces challenges with accuracy and efficiency. To address these issues, we propose a graph-based ANNS algorithm for dense-sparse hybrid vectors. Firstly, we propose a distribution alignment method to improve accuracy, which pre-samples dense and sparse vectors to analyze their distance distribution statistic, resulting in a 1%$\sim$9% increase in accuracy. Secondly, to improve efficiency, we design an adaptive two-stage computation strategy that initially computes dense distances only and later computes hybrid distances. Further, we prune the sparse vectors to speed up the calculation. Compared to naive implementation, we achieve $\sim2.1\times$ acceleration. Thorough experiments show that our algorithm achieves 8.9x$\sim$11.7x throughput at equal accuracy compared to existing hybrid vector search algorithms.

Figures (11)

• Figure 1: Comparison of recall accuracy (Recall@10) on NQ, MA MARCO, HotpotQA datasets, using dense (embedded by BGE model bge_embedding), sparse (embedded by SPLADE model bge_embedding), and hybrid vector retrieval approaches.
• Figure 2: Recall accuracy (Recall@10) of hybrid search with varying dense weights on NQ, MS MARCO, and HotpotQA datasets, embedded by BGE and SPLADE models. After dense and sparse space alignment, the optimal weights shift to around 0.5, achieving higher recall accuracy.
• Figure 3: Left: Comparison of IP distance computing time of sparse vectors with varying number of non-zero elements (dim = 30,000) v.s. dense vectors (dim = 150). Right: Time distribution of hybrid search using naive HNSW algorithm. Sparse calculation accounts for $\sim$61%.
• Figure 4: The distribution of dense distance and sparse distance on NQ, MS MARCO, and HotpotQA datasets embedded by BGE model and SPLADE model.
• Figure 5: Illustration of two-stage graph construction and two-stage search processes.