Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unified and Efficient Approach for Multi-Vector Similarity Search

Binhan Yang, Yuxiang Zeng, Hengxin Zhang, Zhuanglin Zheng, Yunzhen Chi, Yongxin Tong, Ke Xu

Abstract

Multi-Vector Similarity Search is essential for fine-grained semantic retrieval in many real-world applications, offering richer representations than traditional single-vector paradigms. Due to the lack of native multi-vector index, existing methods rely on a filter-and-refine framework built upon single-vector indexes. By treating token vectors within each multi-vector object in isolation and ignoring their correlations, these methods face an inherent dilemma: aggressive filtering sacrifices recall, while conservative filtering incurs prohibitive computational cost during refinement. To address this limitation, we propose MV-HNSW, the first native hierarchical graph index designed for multi-vector data. MV-HNSW introduces a novel edge-weight function that satisfies essential properties (symmetry, cardinality robustness, and query consistency) for graph-based indexing, an accelerated multi-vector similarity computation algorithm, and an augmented search strategy that dynamically discovers topologically disconnected yet relevant candidates. Extensive experiments on seven real-world datasets show that MV-HNSW achieves state-of-the-art search performance, maintaining over 90% recall while reducing search latency by up to 14.0$\times$ compared to existing methods.

Unified and Efficient Approach for Multi-Vector Similarity Search

Abstract

Multi-Vector Similarity Search is essential for fine-grained semantic retrieval in many real-world applications, offering richer representations than traditional single-vector paradigms. Due to the lack of native multi-vector index, existing methods rely on a filter-and-refine framework built upon single-vector indexes. By treating token vectors within each multi-vector object in isolation and ignoring their correlations, these methods face an inherent dilemma: aggressive filtering sacrifices recall, while conservative filtering incurs prohibitive computational cost during refinement. To address this limitation, we propose MV-HNSW, the first native hierarchical graph index designed for multi-vector data. MV-HNSW introduces a novel edge-weight function that satisfies essential properties (symmetry, cardinality robustness, and query consistency) for graph-based indexing, an accelerated multi-vector similarity computation algorithm, and an augmented search strategy that dynamically discovers topologically disconnected yet relevant candidates. Extensive experiments on seven real-world datasets show that MV-HNSW achieves state-of-the-art search performance, maintaining over 90% recall while reducing search latency by up to 14.0 compared to existing methods.

Paper Structure

This paper contains 41 sections, 5 theorems, 8 equations, 8 figures, 4 tables, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM STATEMENT
  3. Basic Concepts
  4. Problem Definition
  5. MULTI-VECTOR INDEX MV-HNSW
  6. Main Idea
  7. Multi-Vector Index Structure
  8. Core Design Principle.
  9. Index Structure.
  10. Index Construction Algorithm
  11. Main Idea.
  12. Algorithm Details.
  13. Complexity Analysis.
  14. Accelerating USim Computation
  15. Main Idea: Filter-and-Refine with Clustering.
  16. ...and 26 more sections

Key Result

Lemma 1

USim is asymmetric, not cardinality-robust, and not query-consistent. $\blacktriangleleft$$\blacktriangleleft$

Figures (8)

  • Figure 1: Overview of our solution MV-HNSW
  • Figure 2: Search performance on all seven real-world datasets (with default query parameter $k=128$)
  • Figure 3: Scalability test with varying dataset cardinality $n$
  • Figure 4: Scalability test with varying the number $c$ of token vectors per query multi-vector
  • Figure 5: Scalability test with varying the dimensionality $d$ of token vectors
  • ...and 3 more figures

Theorems & Definitions (17)

  • Example 1: Document Retrieval
  • Definition 1: Vector Data
  • Definition 2: Single-Vector Similarity Search
  • Definition 3: Multi-vector Data
  • Definition 4: Unified Multi-Vector Similarity Function
  • Example 2
  • Definition 5: Unified Multi-Vector Similarity Search
  • Example 3
  • Definition 6: Symmetry
  • Definition 7: Cardinality Robustness
  • ...and 7 more