Catapults to the Rescue: Accelerating Vector Search by Exploiting Query Locality

TL;DR

CatapultDB is presented, a lightweight mechanism that, for the first time, dynamically determines where to begin the search in an ANN index on the fly, therefore exploiting query locality and matches the efficiency of LSH-based approaches without sacrificing filtering or requiring index reconstruction.

Abstract

Graph-based indexing is the dominant approach for approximate nearest neighbor search in vector databases, offering high recall with low latency across billions of vectors. However, in such indices, the edge set of the proximity graph is only modified to reflect changes in the indexed data, never to adapt to the query workload. This is wasteful: real-world query streams exhibit strong spatial and temporal locality, yet every query must re-traverse the same intermediate hops from fixed or random entry points. We present CatapultDB, a lightweight mechanism that, for the first time, dynamically determines where to begin the search in an ANN index on the fly, therefore exploiting query locality. CatapultDB injects shortcut edges called catapults that connect query regions to frequently visited destination nodes. Catapults are maintained as an additional layer on top of the graph, so the standard vector search algorithm remains unchanged: queries are simply routed to a better starting point when an appropriate catapult exists. This transparent design preserves the full feature set of the underlying system, including filtered search, dynamic insertions, and disk-resident indices. We implement CatapultDB and evaluate it using four workloads with varying amounts of bias. Our experiments show that CatapultDB increases throughput by up to 2.51x compared to DiskANN at equivalent or better recall, matches the efficiency of LSH-based approaches without sacrificing filtering or requiring index reconstruction, and adapts gracefully to workload shifts, unlike cache-based alternatives.

Figures (11)

• Figure 1: Standard beam search for $k=1$ neighbor. The search path is selected greedily, by hopping to the neighbor that minimizes the distance to the query.
• Figure 2: The recall distribution of queries of Proximity when dealing with a static vector database (left) vs. a dynamic one (right).
• Figure 3: A catapult-accelerated beam search for $k=1$ neighbor. Instead of traversing multiple intermediate hops from the entry point, the query is launched directly into its target neighborhood via a catapult edge, significantly reducing the search path length.
• Figure 4: Processing a filtered query in CatapultDB. A filter is associated to the query at runtime, disabling some parts of the proximity graph. Catapults can be conditionally disabled.
• Figure 5: The throughput (in queries per second) and average recall (with four threads, right) of CatapultDB and baselines, while varying the beam width and threads. We use the MedRAG-Zipf workload.