Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bang for the Buck: Vector Search on Cloud CPUs

Leonardo Kuffo, Peter Boncz

TL;DR

The paper addresses the challenge of choosing cloud CPU microarchitectures for vector search across vector databases in high-dimensional spaces of dimension $D$. It systematically benchmarks five architectures (AWS Graviton3/4, Intel Sapphire Rapids Z-series, AMD Zen3/4) across IVF, HNSW, and full-scan workloads with multiple quantization levels, using FAISS and USearch implementations. It shows that performance is strongly influenced by memory-access patterns and distance-kernel efficiency, yielding substantial, index-dependent differences in QPS and the number of queries per dollar ($QP$). The results provide practical guidance for deploying vector search services with maximal bang-for-buck and motivate broader benchmarking across architectures and workloads as CPU generations evolve.

Abstract

Vector databases have emerged as a new type of systems that support efficient querying of high-dimensional vectors. Many of these offer their database as a service in the cloud. However, the variety of available CPUs and the lack of vector search benchmarks across CPUs make it difficult for users to choose one. In this study, we show that CPU microarchitectures available in the cloud perform significantly differently across vector search scenarios. For instance, in an IVF index on float32 vectors, AMD's Zen4 gives almost 3x more queries per second (QPS) compared to Intel's Sapphire Rapids, but for HNSW indexes, the tables turn. However, when looking at the number of queries per dollar (QP$), Graviton3 is the best option for most indexes and quantization settings, even over Graviton4 (Table 1). With this work, we hope to guide users in getting the best "bang for the buck" when deploying vector search systems.

Bang for the Buck: Vector Search on Cloud CPUs

TL;DR

The paper addresses the challenge of choosing cloud CPU microarchitectures for vector search across vector databases in high-dimensional spaces of dimension . It systematically benchmarks five architectures (AWS Graviton3/4, Intel Sapphire Rapids Z-series, AMD Zen3/4) across IVF, HNSW, and full-scan workloads with multiple quantization levels, using FAISS and USearch implementations. It shows that performance is strongly influenced by memory-access patterns and distance-kernel efficiency, yielding substantial, index-dependent differences in QPS and the number of queries per dollar (). The results provide practical guidance for deploying vector search services with maximal bang-for-buck and motivate broader benchmarking across architectures and workloads as CPU generations evolve.

Abstract

Vector databases have emerged as a new type of systems that support efficient querying of high-dimensional vectors. Many of these offer their database as a service in the cloud. However, the variety of available CPUs and the lack of vector search benchmarks across CPUs make it difficult for users to choose one. In this study, we show that CPU microarchitectures available in the cloud perform significantly differently across vector search scenarios. For instance, in an IVF index on float32 vectors, AMD's Zen4 gives almost 3x more queries per second (QPS) compared to Intel's Sapphire Rapids, but for HNSW indexes, the tables turn. However, when looking at the number of queries per dollar (QP$), Graviton3 is the best option for most indexes and quantization settings, even over Graviton4 (Table 1). With this work, we hope to guide users in getting the best "bang for the buck" when deploying vector search systems.
Paper Structure (14 sections, 5 figures, 6 tables)

This paper contains 14 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Approximate Vector Similarity Search
  4. Approximate Indexes
  5. Quantization Techniques
  6. Single Instruction Multiple Data (SIMD)
  7. Bang for the buck
  8. IVF
  9. HNSW
  10. Full Scans
  11. Why the differences across microarchs?
  12. Bang for the buck: Takeways
  13. Discussion
  14. Conclusions and Future Work

Figures (5)

  • Figure 1: Example of an HNSW and IVF index search.
  • Figure 2: Overview of different quantization techniques.
  • Figure 3: QPS and QP$ of cloud instances running queries on a FAISS IVF index without quantization (float32). The Zen4 microarchitecture takes the upper hand in QPS, but Graviton3 takes a slight advantage in QP$.
  • Figure 4: QPS and QP$ of cloud instances running queries on a flat HNSW index (float32). Intel Z takes the upper hand in QPS, but Zen3 and Graviton3 give higher QP$ on high-dimensional vectors. Here, contrary to IVF indexes, Zen4 underperforms.
  • Figure 5: SIMD read bandwidth and access latency (random and sequential) of cache and DRAM in cloud CPUs.