Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods

Rohan Jha, Chunsheng Zuo, Reno Kriz, Benjamin Van Durme

Abstract

While multi-vector retrieval models outperform single-vector models of comparable size in retrieval quality, their practicality is limited by substantially larger index sizes, driven by the additional sequence-length dimension in their document embeddings. Because document embedding size dictates both memory overhead and query latency, compression is essential for deployment. In this work, we present an evaluation of training-free methods targeting the token sequence length, a dimension unique to multi-vector retrieval. Our findings suggest that token merging is strictly superior to token pruning for reducing index size while maintaining retrieval effectiveness.

A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods

Abstract

While multi-vector retrieval models outperform single-vector models of comparable size in retrieval quality, their practicality is limited by substantially larger index sizes, driven by the additional sequence-length dimension in their document embeddings. Because document embedding size dictates both memory overhead and query latency, compression is essential for deployment. In this work, we present an evaluation of training-free methods targeting the token sequence length, a dimension unique to multi-vector retrieval. Our findings suggest that token merging is strictly superior to token pruning for reducing index size while maintaining retrieval effectiveness.
Paper Structure (7 sections, 4 figures)

This paper contains 7 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. Token Pruning
  4. Token Pooling
  5. Experimental Setup
  6. Results
  7. Conclusion

Figures (4)

  • Figure 1: nDCG@10 (left) and Recall@100 (right) relative to Baseline (no compression) averaged across 6 BEIR datasets. Pooling methods (solid lines) consistently degrade much more gracefully than pruning methods (dotted lines) under heavy compression.
  • Figure 2: Dataset-specific breakdowns of nDCG@10 vs. average document length on 6 BEIR datasets. Axes are scaled individually to highlight per-dataset compression characteristics.
  • Figure 3: Dataset-specific breakdowns of Recall@100 vs. average document length on 6 BEIR datasets. Axes are scaled individually to highlight per-dataset compression characteristics.
  • Figure 4: nDCG@10 vs. average document length on 3 CoIR text-to-code retrieval tasks.