Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SINDI: an Efficient Index for Approximate Maximum Inner Product Search on Sparse Vectors

Ruoxuan Li, Xiaoyao Zhong, Jiabao Jin, Peng Cheng, Wangze Ni, Zhitao Shen, Wei Jia, Xiangyu Wang, Heng Tao Shen, Jingkuan Song

Abstract

Sparse vector Maximum Inner Product Search (MIPS) is crucial in multi-path retrieval for Retrieval-Augmented Generation (RAG). Recent inverted index-based and graph-based algorithms have achieved high search accuracy with practical efficiency. However, their performance in production environments is often limited by redundant distance computations and frequent random memory accesses. Furthermore, the compressed storage format of sparse vectors hinders the use of SIMD acceleration. In this paper, we propose the sparse inverted non-redundant distance index (SINDI), which incorporates three key optimizations: (i) Efficient Inner Product Computation: SINDI leverages SIMD acceleration and eliminates redundant identifier lookups, enabling batched inner product computation; (ii) Memory-Friendly Design: SINDI replaces random memory accesses to original vectors with sequential accesses to inverted lists, substantially reducing memory-bound latency. (iii) Vector Pruning: SINDI retains only the high-magnitude non-zero entries of vectors, improving query throughput while maintaining accuracy. We evaluate SINDI on multiple real-world datasets. Experimental results show that SINDI achieves state-of-the-art performance across datasets of varying scales, languages, and models. On the MsMarco dataset, when Recall@50 exceeds 99%, SINDI delivers single-thread query-per-second (QPS) improvements ranging from 4.2$\times$ to 26.4$\times$ compared with SEISMIC and PyANNs. Notably, SINDI has been integrated into Ant Group's open-source vector search library, VSAG.

SINDI: an Efficient Index for Approximate Maximum Inner Product Search on Sparse Vectors

Abstract

Sparse vector Maximum Inner Product Search (MIPS) is crucial in multi-path retrieval for Retrieval-Augmented Generation (RAG). Recent inverted index-based and graph-based algorithms have achieved high search accuracy with practical efficiency. However, their performance in production environments is often limited by redundant distance computations and frequent random memory accesses. Furthermore, the compressed storage format of sparse vectors hinders the use of SIMD acceleration. In this paper, we propose the sparse inverted non-redundant distance index (SINDI), which incorporates three key optimizations: (i) Efficient Inner Product Computation: SINDI leverages SIMD acceleration and eliminates redundant identifier lookups, enabling batched inner product computation; (ii) Memory-Friendly Design: SINDI replaces random memory accesses to original vectors with sequential accesses to inverted lists, substantially reducing memory-bound latency. (iii) Vector Pruning: SINDI retains only the high-magnitude non-zero entries of vectors, improving query throughput while maintaining accuracy. We evaluate SINDI on multiple real-world datasets. Experimental results show that SINDI achieves state-of-the-art performance across datasets of varying scales, languages, and models. On the MsMarco dataset, when Recall@50 exceeds 99%, SINDI delivers single-thread query-per-second (QPS) improvements ranging from 4.2 to 26.4 compared with SEISMIC and PyANNs. Notably, SINDI has been integrated into Ant Group's open-source vector search library, VSAG.

Paper Structure

This paper contains 36 sections, 1 equation, 17 figures, 12 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Definition
  4. Existing Solutions
  5. Full Precision Inverted Index
  6. Value-storing Inverted Index
  7. Efficient Distance Computation
  8. Cache Optimization
  9. Window Switch
  10. Construction and Search
  11. Analysis of Window Size's Impact on Performance
  12. Approximate Inverted Index
  13. Pruning Strategies
  14. Reordering
  15. Experimental Study
  16. ...and 21 more sections

Figures (17)

  • Figure 1: Example of Dense and Sparse Vector Representations and Inner Product Calculations.
  • Figure 2: The Bottleneck of the Graph Index and Inverted Index During Searching Process.
  • Figure 3: Overlap of Inverted List Entries and Common Non-Zero Dimensions in Inner-Product Computation.
  • Figure 4: An Example of Sindi and Query Process.
  • Figure 5: Impact of Window Size on Query Throughput and Memory Accesses.
  • ...and 12 more figures

Theorems & Definitions (1)

  • proof