Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

d-HNSW: A High-performance Vector Search Engine on Disaggregated Memory

Fei Fang, Yi Liu, Chen Qian

Abstract

Efficient vector search is essential for powering large-scale AI applications, such as LLMs. Existing solutions are designed for monolithic architectures where compute and memory are tightly coupled. Recently, disaggregated architecture breaks this coupling by separating compution and memory resources into independently scalable pools to improve utilization. However, applying vector database on disaggregated memory system brings unique challenges to system design due to its graph-based index. We present d-HNSW, the first RDMA-based vector search engine optimized for disaggregated memory systems. d-HNSW preserves HNSW's high accuracy while addressing the new system-level challenges introduced by disaggregation: 1) network inefficiency from pointer-chasing traversals, 2) non-contiguous remote memory layout induced by dynamic insertions, 3) redundant data transfers in batch workloads, and 4) resource underutilization due to sequential execution. d-HNSW tackles these challenges through a set of hardware-algorithm co-designed techniques, including 1) balanced clustering with a lightweight representative index to reduce network round-trips and ensure predictable latency, 2) an RDMA-friendly graph layout that preserves data contiguity under dynamic insertions, 3) query-aware data loading to eliminate redundant fetches across batch queries, and 4) a pipelined execution model that overlaps RDMA transfers with computation to hide network latency and improve throughput. Our evaluation results in a public cloud show that d-HNSW achieves up to < 10-2x query latency and > 100x query throughput compared to other baselines, while maintaining a high recall of 94%.

d-HNSW: A High-performance Vector Search Engine on Disaggregated Memory

Abstract

Efficient vector search is essential for powering large-scale AI applications, such as LLMs. Existing solutions are designed for monolithic architectures where compute and memory are tightly coupled. Recently, disaggregated architecture breaks this coupling by separating compution and memory resources into independently scalable pools to improve utilization. However, applying vector database on disaggregated memory system brings unique challenges to system design due to its graph-based index. We present d-HNSW, the first RDMA-based vector search engine optimized for disaggregated memory systems. d-HNSW preserves HNSW's high accuracy while addressing the new system-level challenges introduced by disaggregation: 1) network inefficiency from pointer-chasing traversals, 2) non-contiguous remote memory layout induced by dynamic insertions, 3) redundant data transfers in batch workloads, and 4) resource underutilization due to sequential execution. d-HNSW tackles these challenges through a set of hardware-algorithm co-designed techniques, including 1) balanced clustering with a lightweight representative index to reduce network round-trips and ensure predictable latency, 2) an RDMA-friendly graph layout that preserves data contiguity under dynamic insertions, 3) query-aware data loading to eliminate redundant fetches across batch queries, and 4) a pipelined execution model that overlaps RDMA transfers with computation to hide network latency and improve throughput. Our evaluation results in a public cloud show that d-HNSW achieves up to < 10-2x query latency and > 100x query throughput compared to other baselines, while maintaining a high recall of 94%.
Paper Structure (32 sections, 18 figures, 2 tables, 4 algorithms)

This paper contains 32 sections, 18 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background, Opportunities, and Challenges
  3. Vector similarity search
  4. RDMA-based disaggregated memory
  5. Opportunity and Challenges
  6. d-HNSW Design Overview
  7. d-HNSW Design
  8. Representative index caching
  9. Index Balancer
  10. Representative Metadata
  11. RDMA-friendly graph index storage layout in remote memory
  12. Query-aware batched data loading
  13. Orchestrating vector search and network data transfer pipeline
  14. Index Rebuild Mechanism
  15. Performance Modeling and Analysis
  16. ...and 17 more sections

Figures (18)

  • Figure 1: Graph-based vector search index: HNSW.
  • Figure 2: CDF of HNSW pointer-chasing operations per query for different search candidate list size on SIFT1M.
  • Figure 3: Search latency after 10K insertions.
  • Figure 4: CDF of fetch count per partition in one batch.
  • Figure 5: Overview of d-HNSW.
  • ...and 13 more figures