Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bi-metric Framework for Fast Similarity Search

Haike Xu, Sandeep Silwal, Piotr Indyk

TL;DR

This work proposes a new "bi-metric" framework for designing nearest neighbor data structures that achieves a considerably better accuracy-efficiency tradeoff than the alternatives, such as re-ranking.

Abstract

We propose a new "bi-metric" framework for designing nearest neighbor data structures. Our framework assumes two dissimilarity functions: a ground-truth metric that is accurate but expensive to compute, and a proxy metric that is cheaper but less accurate. In both theory and practice, we show how to construct data structures using only the proxy metric such that the query procedure achieves the accuracy of the expensive metric, while only using a limited number of calls to both metrics. Our theoretical results instantiate this framework for two popular nearest neighbor search algorithms: DiskANN and Cover Tree. In both cases we show that, as long as the proxy metric used to construct the data structure approximates the ground-truth metric up to a bounded factor, our data structure achieves arbitrarily good approximation guarantees with respect to the ground-truth metric. On the empirical side, we apply the framework to the text retrieval problem with two dissimilarity functions evaluated by ML models with vastly different computational costs. We observe that for almost all data sets in the MTEB benchmark, our approach achieves a considerably better accuracy-efficiency tradeoff than the alternatives, such as re-ranking.

A Bi-metric Framework for Fast Similarity Search

TL;DR

This work proposes a new "bi-metric" framework for designing nearest neighbor data structures that achieves a considerably better accuracy-efficiency tradeoff than the alternatives, such as re-ranking.

Abstract

We propose a new "bi-metric" framework for designing nearest neighbor data structures. Our framework assumes two dissimilarity functions: a ground-truth metric that is accurate but expensive to compute, and a proxy metric that is cheaper but less accurate. In both theory and practice, we show how to construct data structures using only the proxy metric such that the query procedure achieves the accuracy of the expensive metric, while only using a limited number of calls to both metrics. Our theoretical results instantiate this framework for two popular nearest neighbor search algorithms: DiskANN and Cover Tree. In both cases we show that, as long as the proxy metric used to construct the data structure approximates the ground-truth metric up to a bounded factor, our data structure achieves arbitrarily good approximation guarantees with respect to the ground-truth metric. On the empirical side, we apply the framework to the text retrieval problem with two dissimilarity functions evaluated by ML models with vastly different computational costs. We observe that for almost all data sets in the MTEB benchmark, our approach achieves a considerably better accuracy-efficiency tradeoff than the alternatives, such as re-ranking.
Paper Structure (29 sections, 10 theorems, 5 equations, 10 figures, 1 table, 3 algorithms)

This paper contains 29 sections, 10 theorems, 5 equations, 10 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our results
  3. Related Work
  4. Graph-based algorithms for similarity search
  5. Data analysis with proxy metrics
  6. Preliminaries
  7. Problem formulation:
  8. Assumptions about metrics:
  9. Theoretical analysis
  10. DiskANN
  11. Preliminaries for DiskANN
  12. The main theorem.
  13. Experiments
  14. Experiment Setup
  15. Methods
  16. ...and 14 more sections

Key Result

Theorem 1.1

Given a dataset $X$ of $n$ points, $\mathtt{Alg} \in \{\text{DiskANN}, \text{Cover Tree}\}$, and a fixed metric $d$, let $S_{\mathtt{Alg}}(n, \varepsilon, \lambda_d)$ and $Q_{\mathtt{Alg}}(\varepsilon, \lambda_d)$ denote the space and query complexity respectively of the standard datastructure for $

Figures (10)

  • Figure 1: Results for 15 MTEB Retrieval datasets. The x-axis is the number of expensive distance function calls. The y-axis is the NDCG@10 score. The cheap model is "bge-micro-v2", the expensive model is "SFR-Embedding-Mistral", and the nearest neighbor search algorithm used is DiskANN.
  • Figure 2: HotpotQA test results for different models as the distance proxy. Blue / skyblue / cyan curves represent Bi-metric (our method) with bge-micro / gte-small / bge-base models. Red / rose / magenta curves represent Bi-metric (baseline) with bge-micro / gte-small / bge-base models
  • Figure 3: HotpotQA test results for different search initializations for the second-stage search of Bi-metric (our method). Blue / purple / brown / green curves represent initializing our second-stage search with top-$\mathcal{Q}/2$, top-100, top-1, or the default vertex.
  • Figure 4: Results for 15 MTEB Retrieval datasets. The x-axis is the number of expensive distance function calls. The y-axis is the Recall@10 score. The cheap model is "bge-micro-v2", the expensive model is "SFR-Embedding-Mistral", and the nearest neighbor search algorithm used is DiskANN.
  • Figure 5: Results for 15 MTEB Retrieval datasets. The x-axis is the number of expensive distance function calls. The y-axis is the NDCG@10 score. The cheap model is "gte-small", the expensive model is "SFR-Embedding-Mistral", and the nearest neighbor search algorithm used is DiskANN.
  • ...and 5 more figures

Theorems & Definitions (20)

  • Theorem 1.1: Summary, see Theorems \ref{['thm:diskann_main']} and \ref{['thm:ct_main']}
  • Definition 2.1
  • Definition 2.2: Doubling Dimension
  • Definition 3.1: $\alpha$-shortcut reachability NEURIPS2023_worstcase
  • Theorem 3.2: NEURIPS2023_worstcase
  • Theorem 3.3: Theorem $3.4$ in NEURIPS2023_worstcase
  • Theorem 3.4
  • Lemma 3.5
  • proof
  • proof : Proof of Theorem \ref{['thm:diskann_main']}
  • ...and 10 more