Efficient Vector Search on Disaggregated Memory with d-HNSW
Yi Liu, Fei Fang, Chen Qian
TL;DR
The paper addresses the challenge of scalable vector search when datasets exceed a single machine's memory by proposing d-HNSW, an RDMA-based disaggregated memory vector search engine. It introduces a two-tier indexing strategy consisting of a lightweight meta-HNSW for coarse routing and multiple sub-HNSWs for fine-grained search, complemented by representative index caching, an RDMA-friendly index layout, and query-aware batched loading to minimize network transfers. The authors implement a prototype and demonstrate substantial latency improvements (up to 117x, and up to 121x in some benchmarks) with recall around 0.86 on common datasets, validating the practicality of disaggregated-memory vector search. The work contributes a practical architecture for high-throughput vector retrieval in data centers, potentially enabling scalable retrieval-augmented generation and other AI workloads that rely on large embedding spaces.
Abstract
Efficient vector query processing is critical to enable AI applications at scale. Recent solutions struggle with growing vector datasets that exceed single-machine memory capacity, forcing unnecessary data movement and resource underutilization in monolithic architectures. We present d-HNSW, the first disaggregated vector similarity search engine for RDMA-based remote memory systems that achieves high performance while supporting fast data indexing with low network communication overhead. The core of d-HNSW is a novel disaggregation of the graph-based vector indexing data structure HNSW. It exploits the characteristics of greedy searching in HNSW to efficiently coordinate data transfers from the memory pool to the compute pool while serving data requests. Specifically, it leverages three ideas: (i) Representative index caching, a lightweight index constructed from a sampled subset of data, is cached in the compute pool to reduce frequent access to critical components of the hierarchical graph-based index, (ii) RDMA-friendly data layout design to reduce the networking round trips incurred by vector query and insertion and (iii) batched query-aware data loading to reduce bandwidth usage on data transfer between pools, addressing the limited cache capacity in compute nodes. We evaluate d-HNSW with extensive benchmarking datasets. The experimental results show that d-HNSW outperforms Naive d-HNSW implementation by up to 117x in latency while maintaining recall as 0.87 in dataset SIFT1M@1.