Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficiency Optimizations for Superblock-based Sparse Retrieval

Parker Carlson, Wentai Xie, Rohil Shah, Tao Yang

TL;DR

This work tackles efficient first-stage sparse retrieval by introducing Lightweight Superblock Pruning (LSP), a top-$\gamma$-inclusive pruning scheme that guarantees visiting a fixed number of high-signal superblocks while omitting the heavy average-bound safeguards. By pairing LSP with compact data structures (SIMDBP-256*) and aggressive 4-bit quantization for block/superblock weights, the approach achieves substantial speedups (e.g., up to 17× faster than SP and 12× faster than BMP in key settings) while preserving near-safe recall across MS MARCO and BEIR. The method demonstrates strong zero-shot generalization across SPLADE families and BEIR datasets, with robust parameter configurations that avoid grid searches. Overall, LSP provides a simple, scalable, and dataset-robust alternative for efficient sparse retrieval on CPU, with clear compression benefits and practical deployment guidance.

Abstract

Learned sparse retrieval (LSR) is a popular method for first-stage retrieval because it combines the semantic matching of language models with efficient CPU-friendly algorithms. Previous work aggregates blocks into "superblocks" to quickly skip the visitation of blocks during query processing by using an advanced pruning heuristic. This paper proposes a simple and effective superblock pruning scheme that reduces the overhead of superblock score computation while preserving competitive relevance. It combines this scheme with a compact index structure and a robust zero-shot configuration that is effective across LSR models and multiple datasets. This paper provides an analytical justification and evaluation on the MS MARCO and BEIR datasets, demonstrating that the proposed scheme can be a strong alternative for efficient sparse retrieval.

Efficiency Optimizations for Superblock-based Sparse Retrieval

TL;DR

This work tackles efficient first-stage sparse retrieval by introducing Lightweight Superblock Pruning (LSP), a top--inclusive pruning scheme that guarantees visiting a fixed number of high-signal superblocks while omitting the heavy average-bound safeguards. By pairing LSP with compact data structures (SIMDBP-256*) and aggressive 4-bit quantization for block/superblock weights, the approach achieves substantial speedups (e.g., up to 17× faster than SP and 12× faster than BMP in key settings) while preserving near-safe recall across MS MARCO and BEIR. The method demonstrates strong zero-shot generalization across SPLADE families and BEIR datasets, with robust parameter configurations that avoid grid searches. Overall, LSP provides a simple, scalable, and dataset-robust alternative for efficient sparse retrieval on CPU, with clear compression benefits and practical deployment guidance.

Abstract

Learned sparse retrieval (LSR) is a popular method for first-stage retrieval because it combines the semantic matching of language models with efficient CPU-friendly algorithms. Previous work aggregates blocks into "superblocks" to quickly skip the visitation of blocks during query processing by using an advanced pruning heuristic. This paper proposes a simple and effective superblock pruning scheme that reduces the overhead of superblock score computation while preserving competitive relevance. It combines this scheme with a compact index structure and a robust zero-shot configuration that is effective across LSR models and multiple datasets. This paper provides an analytical justification and evaluation on the MS MARCO and BEIR datasets, demonstrating that the proposed scheme can be a strong alternative for efficient sparse retrieval.
Paper Structure (11 sections, 6 equations, 3 figures, 9 tables)

This paper contains 11 sections, 6 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Block/Superblock Pruning and Design Consideration
  4. Lightweight Superblock Pruning
  5. Pruning with top-$\gamma$ superblock inclusion
  6. Choice of $\gamma$ for top superblock inclusion
  7. Index compression
  8. Experimental Studies
  9. Effectiveness on MS MARCO and BEIR
  10. Compression evaluation
  11. Conclusion

Figures (3)

  • Figure 3: Traversal with superblock pruning in LSP
  • Figure 4: Probability of containing top-$k$ documents at different sorted top superblock positions with $1\leq \gamma \leq 3000$
  • Figure 5: (a) Flat inverted index structure for documents within a block. (b) SIMDBP-256* data structure layout for storing maximum term weights for each block or superblock