Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing SSD-Resident Graph Indexing for High-Throughput Vector Search

Weichen Zhao, Yuncheng Lu, Yao Tian, Hao Zhang, Jiehui Li, Minghao Zhao, Yakun Li, Weining Qian

TL;DR

VeloANN utilizes hierarchical compression and affinity-based data placement scheme to co-locate related vectors within the same page, effectively reducing fragmentation and over-fetching and incorporates asynchronous prefetching and a beam-aware search strategy to prioritize cached data, ultimately improving overall search efficiency.

Abstract

Graph-based approximate nearest neighbor search (ANNS) methods (e.g., HNSW) have become the de facto state of the art for their high precision and low latency. To scale beyond main memory, recent out-of-memory ANNS systems leverage SSDs to store large vector indexes. However, they still suffer from severe CPU underutilization and read amplification (i.e., storage stalls) caused by limited access locality during graph traversal. We present VeloANN, which mitigates storage stalls through a locality-aware data layout and a coroutine-based asynchronous runtime. VeloANN utilizes hierarchical compression and affinity-based data placement scheme to co-locate related vectors within the same page, effectively reducing fragmentation and over-fetching. We further design a record-level buffer pool, where each record groups the neighbors of a vector; by persistently retaining hot records in memory, it eliminates excessive page swapping under constrained memory budgets. To minimize CPU scheduling overheads during disk I/O interruptions, VeloANN employs a coroutine-based asynchronous runtime for lightweight task scheduling. On top of this, it incorporates asynchronous prefetching and a beam-aware search strategy to prioritize cached data, ultimately improving overall search efficiency. Extensive experiments show that VeloANN outperforms state-of-the-art disk-based ANN systems by up to 5.8x in throughput and 3.25x in latency reduction, while achieving 0.92x the throughput of in-memory systems using only 10% of their memory footprint.

Optimizing SSD-Resident Graph Indexing for High-Throughput Vector Search

TL;DR

VeloANN utilizes hierarchical compression and affinity-based data placement scheme to co-locate related vectors within the same page, effectively reducing fragmentation and over-fetching and incorporates asynchronous prefetching and a beam-aware search strategy to prioritize cached data, ultimately improving overall search efficiency.

Abstract

Graph-based approximate nearest neighbor search (ANNS) methods (e.g., HNSW) have become the de facto state of the art for their high precision and low latency. To scale beyond main memory, recent out-of-memory ANNS systems leverage SSDs to store large vector indexes. However, they still suffer from severe CPU underutilization and read amplification (i.e., storage stalls) caused by limited access locality during graph traversal. We present VeloANN, which mitigates storage stalls through a locality-aware data layout and a coroutine-based asynchronous runtime. VeloANN utilizes hierarchical compression and affinity-based data placement scheme to co-locate related vectors within the same page, effectively reducing fragmentation and over-fetching. We further design a record-level buffer pool, where each record groups the neighbors of a vector; by persistently retaining hot records in memory, it eliminates excessive page swapping under constrained memory budgets. To minimize CPU scheduling overheads during disk I/O interruptions, VeloANN employs a coroutine-based asynchronous runtime for lightweight task scheduling. On top of this, it incorporates asynchronous prefetching and a beam-aware search strategy to prioritize cached data, ultimately improving overall search efficiency. Extensive experiments show that VeloANN outperforms state-of-the-art disk-based ANN systems by up to 5.8x in throughput and 3.25x in latency reduction, while achieving 0.92x the throughput of in-memory systems using only 10% of their memory footprint.
Paper Structure (22 sections, 2 equations, 14 figures, 3 tables, 2 algorithms)

This paper contains 22 sections, 2 equations, 14 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Approximate Nearest Neighbor Search
  4. Graph-based ANNS Index
  5. Asynchronous Execution Framework
  6. Larger-than-Memory Database Techniques
  7. VeloANN Design
  8. Coroutine-based ANNS Execution
  9. Buffer Pool Management
  10. Physical Index Layout
  11. Record Co-Placement Mechanism
  12. Cache-Aware ANNS Procedure
  13. Prefetching
  14. Cache-aware Beam Search
  15. Evaluation
  16. ...and 7 more sections

Figures (14)

  • Figure 1: Query throughput comparison on the GIST1M dataset among an in-memory index (Vamana jayaram2019diskann), on-disk indexes (DiskANN jayaram2019diskann, Starling wang2024starling, PipeANN guo2025achieving), and our VeloANN. The memory budget is 20% of the disk index size.
  • Figure 2: Overview of asynchronous execution in VeloANN.
  • Figure 3: Coroutine-based ANNS Execution. Each thread has its own scheduler that batches incoming queries and interleaves multiple query coroutines.
  • Figure 4: Access frequency distributions at node- and page-level granularities for DiskANN on Sift1M and Gist1M.
  • Figure 5: The state transitions of record states.
  • ...and 9 more figures