Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SpANNS: Optimizing Approximate Nearest Neighbor Search for Sparse Vectors Using Near Memory Processing

Tianqi Zhang, Flavio Ponzina, Tajana Rosing

TL;DR

This work targets the bottleneck of sparse ANNS in information retrieval by introducing SpANNS, a near-memory processing accelerator on a CXL Type-2 platform. The design combines a two-level hybrid inverted index (level-1 content-based filtering and level-2 silhouette-based clustering) with a forward index, and distributes computation across L2Inv and F-Idx DIMMs to minimize host data movement. Key contributions include a novel NMP-friendly indexing scheme, a detailed architecture with a Type-2 controller and compute-enabled DIMMs, and runtime optimizations for sparsity-aware distance computation and load balancing. Empirically, SpANNS achieves up to 15–22x speedups over CPU baselines and up to ~1500x over GPU-exhaustive search while maintaining high recall, demonstrating practical scalability for hybrid IR pipelines that rely on sparse embeddings.

Abstract

Approximate Nearest Neighbor Search (ANNS) is a fundamental operation in vector databases, enabling efficient similarity search in high-dimensional spaces. While dense ANNS has been optimized using specialized hardware accelerators, sparse ANNS remains limited by CPU-based implementations, hindering scalability. This limitation is increasingly critical as hybrid retrieval systems, combining sparse and dense embeddings, become standard in Information Retrieval (IR) pipelines. We propose SpANNS, a near-memory processing architecture for sparse ANNS. SpANNS combines a hybrid inverted index with efficient query management and runtime optimizations. The architecture is built on a CXL Type-2 near-memory platform, where a specialized controller manages query parsing and cluster filtering, while compute-enabled DIMMs perform index traversal and distance computations close to the data. It achieves 15.2x to 21.6x faster execution over the state-of-the-art CPU baselines, offering scalable and efficient solutions for sparse vector search.

SpANNS: Optimizing Approximate Nearest Neighbor Search for Sparse Vectors Using Near Memory Processing

TL;DR

This work targets the bottleneck of sparse ANNS in information retrieval by introducing SpANNS, a near-memory processing accelerator on a CXL Type-2 platform. The design combines a two-level hybrid inverted index (level-1 content-based filtering and level-2 silhouette-based clustering) with a forward index, and distributes computation across L2Inv and F-Idx DIMMs to minimize host data movement. Key contributions include a novel NMP-friendly indexing scheme, a detailed architecture with a Type-2 controller and compute-enabled DIMMs, and runtime optimizations for sparsity-aware distance computation and load balancing. Empirically, SpANNS achieves up to 15–22x speedups over CPU baselines and up to ~1500x over GPU-exhaustive search while maintaining high recall, demonstrating practical scalability for hybrid IR pipelines that rely on sparse embeddings.

Abstract

Approximate Nearest Neighbor Search (ANNS) is a fundamental operation in vector databases, enabling efficient similarity search in high-dimensional spaces. While dense ANNS has been optimized using specialized hardware accelerators, sparse ANNS remains limited by CPU-based implementations, hindering scalability. This limitation is increasingly critical as hybrid retrieval systems, combining sparse and dense embeddings, become standard in Information Retrieval (IR) pipelines. We propose SpANNS, a near-memory processing architecture for sparse ANNS. SpANNS combines a hybrid inverted index with efficient query management and runtime optimizations. The architecture is built on a CXL Type-2 near-memory platform, where a specialized controller manages query parsing and cluster filtering, while compute-enabled DIMMs perform index traversal and distance computations close to the data. It achieves 15.2x to 21.6x faster execution over the state-of-the-art CPU baselines, offering scalable and efficient solutions for sparse vector search.
Paper Structure (17 sections, 1 equation, 7 figures, 2 tables)

This paper contains 17 sections, 1 equation, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works: Inverted index-based ANNS
  3. SpANNS Overview
  4. Hybrid inverted index for sparse vector ANNS
  5. SpANNS Near memory accelerator
  6. Architecture overview
  7. SpANNS Dataflow
  8. SpANNS Design Details
  9. Query-aware Runtime Optimization
  10. Results
  11. Experimental Setup
  12. SpANNS Performance
  13. Load Balancing Analysis
  14. Early Termination for Query Optimization
  15. Area and Power Overhead Analysis
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Sparse Vector ANNS in the Vector Search Engine for Information Retrieval Pipelines.
  • Figure 2: Illustration of Inverted Index Approaches: (a) Content-Based, (b) Clustering-Based, and (c) Hybrid Design.
  • Figure 3: Hybrid Inverted Index for Sparse Vector ANNS
  • Figure 4: Architecture of SpANNS: (a) Dataflow and Components, (b) Silhouette Check Logic in Type-2 Controller, (c) Configurable top-K priority queue, (d) Compute Unit for Data Compact in Forward Index Ranks, and (e) Distance Calculation Module in Forward Index DIMMs.
  • Figure 5: Throughput of SpANNS vs. GPU cusparse, ANNA ANNA, WAND knowhere, PyANNS pyanns, and Seismic seismic.
  • ...and 2 more figures